POEM 101: Input checking utility

[lamkina]

Add an input checking tool to help debug how inputs are being set for complicated models.

[robfalck]

Thanks for the suggestion! I'm trying to think of a way we might be able to achieve the desired result by logging calls to set_val or set_input_defaults. Perhaps rather than default/intermedite/user we could track which values were set outside of the add_input/add_output call, and track the line/file in which the value was set. That might be a little more immediate in providing utility to the user.

[lamkina]

I agree that a more fine-grained logging would be useful. Keeping the logging calls related to the syntax and line/file seems reasonable and would point the user directly to the location in the code.

[robfalck]

I have a notional implementation of this at my branch here: https://github.com/robfalck/OpenMDAO/tree/set_val_tracking

This adds a new metadata entry to variables, val_info.

Problem.set_val, Problem.__setitem__, and System.set_val obtain the file and line number of the caller and save that information into the val_info field of the variable whose value is being set.

Currently this information is only available in the inputs report, but we can extend this to list_inputs, list_outputs, and list_vars.

So give this branch a shot and see if the info in reports/inputs.html and see if this is useful to you.
It's pretty rough right now. Theres no special handling for discretes and I'm not sure how well this would perform on MPI.

[sabakhshi]

Hi Rob,

I was looking forward to trying out this feature as the POEM was inspired by an issue I was having. I found a great opportunity to try it out on a coupled Dymos-OAS problem where I connect the roll moment(CMx) output by OAS to the Dymos dynamics model. Note that I've put all my OAS instances inside a component as an OpenMDAO sub-problem.

In my N2 diagram I noticed that my CMx outputs from OAS were not connected causing my optimization to fail as the dynamics model always saw a CMx of 1. While I knew where the issue was I tried using your branch to see if I could use it to match my initial diagnosis.

Unfortunately, generating the reports threw an error and I actually do not think it has much to do with this POEM but rather how I have my control variable(delta) connected to OAS appears to have broken the report writing logic in OpenMDAO.

I've done some debugging and I have a good idea on why this is the case. I'll go ahead and explain it here and maybe we can move it to an issue later. I've developed a framework to enable multi-section wing morphing in OAS. In this simple test case however I'm just using it on a simple rectangular wing with two sections that are being deflected in opposite directions using a single twist B-spline control point on each section to emulate aileron-type control. As result my control input delta is transformed from a length 80 vector to a (80,2) size array(this is done by separate unrelated component I've made). Each row of this (80,2) matrix is connected to it's corresponding instance of OAS and each column is connected to it's corresponding section.

As seen in the screenshot above the twist B-spline components in the OAS instances take a (1,1) array as input and this value is sliced out of the (80,2) array. This approach has been tested and I've been able to successfully converge several couple Dymos-OAS optimization problems using it.

The issue however comes when the user tries to write reports or N2 diagrams. N2 diagrams works but always spam the users console with the following warning.

/home/safa/OpenMDAO/openmdao/visualization/n2_viewer/n2_viewer.py:114: OpenMDAOWarning:attribute 'shape' of 'numpy.generic' objects is not writable

I didn't think much of this at first as the N2 diagram was still being generated successfully. However, when I first turned on reports so I could test this POEM out OpenMDAO being throwing this error.

File "/home/safa/OpenMDAO/openmdao/visualization/inputs_report/inputs_report.py", line 157, in _run_inputs_report inputs_report(prob, display=False, outfile=path, File "/home/safa/OpenMDAO/openmdao/visualization/inputs_report/inputs_report.py", line 103, in inputs_report val = model.get_val(target, get_remote=True, from_src=not src.startswith('_auto_ivc.')) File "/home/safa/OpenMDAO/openmdao/core/system.py", line 5467, in get_val return caller._get_input_from_src(name, abs_names, conns, units=simp_units, File "/home/safa/OpenMDAO/openmdao/core/system.py", line 5935, in _get_input_from_src val.shape = vshape AttributeError: attribute 'shape' of 'numpy.generic' objects is not writable

This looks similar to the warning seen before in the N2 diagram. I ran the debugger and the issue was indeed in the _get_input_from_src function inside /openmdao/core/system.py and occurs with the "delta" variable I explained earlier.

My apologies for the code screenshot but I wanted to get the debugger output in there to make the issue clear. The issue occurs when the shape of the input is (1,1) as I showed earlier but when you slice the array to get val the output is a float. You can't assign the shape (1,1) to float so we get an error.

I am happy to open and issue for this or to make a PR to fix it but I wanted to consult with the OpenMDAO team first. This issue might seem pretty obscure but I think it's likely to occur anytime one tries to slice an array into a single control point B-spline which is a common input with shape (1,1).

[robfalck]

Thanks for the thorough report. I'll look into it

[robfalck]

@sabakhshi is a version of your script available for testing?
What version of numpy are you using?
This feels like it might be an instance where we're assigning a different but compatible shape, but I haven't seen that manifest with the error about numpy.generic before.

[robfalck]

I'm not sure right now under which circumstances that val will show up as a numpy.generic.
Debugging these things without the actual script is difficult, but try replacing lines ~6013 to ~6024 of system.py with the following:

                    if src_indices._flat_src:
                        val = val.ravel()[src_indices.flat()]
                        # if at component level, just keep shape of the target and don't flatten
                        if not flat and not is_prom:
                            val = np.reshape(val, vmeta['shape'])
                    else:
                        val = val[src_indices()]
                        if vshape is not None and val.shape != vshape:
                            val = np.reshape(val, vshape)
                        elif not is_prom and vmeta is not None and val.shape != vmeta['shape']:
                            val = np.reshape(val, vmeta['shape'])

I've put this small change onto a branch https://github.com/robfalck/OpenMDAO/pull/new/n2_np_generic_issue

If you merge that in on top of my set_val_tracking branch, I'd like to see if it resolves the issue.

[sabakhshi]

Hi Rob,

Thanks for looking into this. I am using Numpy 1.26.4 Unfortunately, I can't share the script I am experiencing this issue with. However, I can report that your fix did allow me to generate reports without any errors so thank you for providing that. However, the warnings regarding the N2 diagram still continue to appear suggesting that it might be an unrelated issue? Regardless, I'll look into providing a script that can reproduce this issue for you.

Thanks

[robfalck]

Thanks. Appreciate the thoroughness of the report :)

[robfalck]

One thing that could also help me get this fixed is to replace the issue_warning at n2_viewer.py with a bare raise statement that will reraise the error. This will cause a hard error but should at least point to the offending line of code.

Also, I pushed up some more changes to that branch that hopefully squashed the errors that the n2 diagram was hitting.