[fx_importer] Convert non-persistent buffers lifted as tensor constants

[sjain-stanford]

The investigation is largely recorded in #2881, but this change allows us to capture non-persistent buffers that were lifted as tensor constants (after pytorch/pytorch#118969 landed in upstream PyTorch), and propagate them to Torch dialect as "frozen" torch.vtensor.literal. I believe this patch should work with both nightly and stable PyTorch, but will let CI confirm the same. Thanks @stellaraccident for the valuable pointers and guidance.

[stellaraccident]

If the CI passes, this lgtm and can land. Once it does, let's close the original PR.

[stellaraccident]

You'll need to hasattr(..., "constants") to guard the new clause: " AttributeError: 'ExportedProgram' object has no attribute 'constants'"

If doing that, I would probably make it fully conditional with a note about where the change happened: If there is no "constants" attribute, consult the "state_dict". Otherwise, only look at "constants". Future us will thank us because the freezing of states will be something we want to undo eventually (just like the upstream patch had to fix this).

[sjain-stanford]

If doing that, I would probably make it fully conditional with a note about where the change happened: If there is no "constants" attribute, consult the "state_dict". Otherwise, only look at "constants". Future us will thank us because the freezing of states will be something we want to undo eventually (just like the upstream patch had to fix this).

Done. Just a clarification that we will need to continue using state_dict for parameters (irrespective of how buffers are handled). Is that correct?

[stellaraccident]

If doing that, I would probably make it fully conditional with a note about where the change happened: If there is no "constants" attribute, consult the "state_dict". Otherwise, only look at "constants". Future us will thank us because the freezing of states will be something we want to undo eventually (just like the upstream patch had to fix this).

Done. Just a clarification that we will need to continue using state_dict for parameters (irrespective of how buffers are handled). Is that correct?

I think that how you have this now gets this in the safe state. If we were unconditionally looking at state_dict, then we would likely be lifting things that future-us actually wants to be treating as mutable, model-level things. Many systems are just figuring out what to do about that, but better to have this do the right thing at the front door for now.