Latest docker ooanalyzer crashes on executable

[janbbeck]

ooanalyzer asked me to report this :P

PLOG[FATAL]: Refusing to backtrack into reasoningLoop to fix an upstream problem because backtrackForUpstream/0 is not set.
PLOG[FATAL]: This likely indicates that there is a problem with the OO rules.
PLOG[FATAL]: Please report this failure to the Pharos developers!
 [618] backtrace(100) at /usr/local/lib/swipl/library/prolog_stack.pl:487
 [617] throw_with_backtrace(error(system_error(upstreamProblem))) at /usr/local/share/pharos/prolog/oorules/util.pl:183
   [3] solve_internal at /usr/local/share/pharos/prolog/oorules/setup.pl:637
   [2] catch(user:solve_internal,_635579612,user:(...,...)) at /usr/local/lib/swipl/boot/init.pl:482
   [1] solve(ooanalyzer_tool) at /usr/local/share/pharos/prolog/oorules/setup.pl:583
   [0] <meta call>

Note: some frames are missing due to last-call optimization.
Re-run your program in debug mode (:- debug.) to get more detail.
OOAN[FATAL]: SWI Prolog Error: Unknown message: error(system_error(upstreamProblem))
OOAN[ERROR]: Failed to import object oriented results into Pharos.
OPTI[ERROR]: No C++ classes were detected in the program.
OPTI[INFO ]: OOAnalyzer analysis complete.

NFS_Mill.zip

[sei-eschwartz]

Thanks for reporting this. Could you upload your log file and facts file? It would save me some time.

[sei-eschwartz]

Also what command line are you using?

[janbbeck]

I used
/usr/local/bin/ooanalyzer ./NFS_Mill.exe --json NFS_Mill.json --timeout 1000000 --maximum-memory 1500000 --partitioner-timeout 1000000 --log='APID(all)'
where can I find the log and facts files? They are not in the directory where I executed the command.

[sei-eschwartz]

If you do not request them to be saved, they won't. You may want to read through this since your executable is huge, and you probably don't want to restart from scratch each time: https://github.com/cmu-sei/pharos/blob/master/share/prolog/oorules/README.md

You may be able to recover logs with docker logs though depending on how you ran the command.

[sei-eschwartz]

I was able to reproduce the problem.

Guessing factConstructor(0x727290).
... a lot of reasoning happens here ...
insanityClassSizeInvalid failed: Class=0xe19f28 LTESize=0x14 GTESize=0x20
Guessing factNOTConstructor(0x727290).
reasonNOTConstructor_F(0x712f70).
Contradictory information about constructor: factConstructor(0x712f70) but reasonNOTConstructor(0x712f70)

[sei-eschwartz]

From cat NFS_Mill.exe.results.log | fgrep -e Guess -e 0xe19f28

Guessing factConstructor(0x727290).
reasonMergeVFTables_A(destructor, 0xe19f28, 0x727290, 0xe19f28, 0, factVFTableWrite(0x727298, 0x727290, 0, 0xe19f28)).
reasonMergeVFTables_A(constructor, 0xe19f28, 0x727290, 0xe19f28, 0, factVFTableWrite(0x727298, 0x727290, 0, 0xe19f28)).
Concluding mergeVFTables(0xe19f28, 0x727290).
Merging class 0x727290 into 0xe19f28 ...
Retracting factClassCallsMethod(0x727290, 0x712f70) and asserting factClassCallsMethod(0xe19f28, 0x712f70) ...
Retracting factClassSizeGTE(0x727290, 0x20) and asserting factClassSizeGTE(0xe19f28, 0x20) ...
Retracting factClassSizeLTE(0x727290, 0x14) and asserting factClassSizeLTE(0xe19f28, 0x14) ...
Retracting factClassSizeLTE(0x727290, 0x20) and asserting factClassSizeLTE(0xe19f28, 0x20) ...

It looks like 0x727290 should already fail sanity checks before the guess.

[sei-eschwartz]

Ignore previous message... I wasn't printing enough.

Guessing factConstructor(0x727290).
Concluding factNOTRealDestructor(0x727290).
reasonClassSizeGTE_D(0x727290, 0x20).
reasonClassSizeGTE_D(0x727290, 0x20).
Concluding factClassSizeGTE(0x727290, 0x20).
reasonClassSizeLTE_C(sv_12071439086702097819, 0x727290, 0x14).
reasonClassSizeLTE_C(sv_1389056183026405998, 0x727290, 0x20).
reasonClassSizeLTE_C(sv_4978211927022902667, 0x727290, 0x20).
Concluding factClassSizeLTE(0x727290, 0x14).
Concluding factClassSizeLTE(0x727290, 0x20).
Concluding factClassSizeLTE(0x727290, 0xfffffff).

[sei-eschwartz]

reasonClassSizeGTE_D:

% The given class (associated with the constructor) is certain to be of this exact size.  The                                                                                                 
% reasoning is that we're able to track an allocation site with a known size to the constructor                                                                                               
% associated with the class.  There's a small bit of ambiguity about what the compiler will                                                                                                   
% generate for arrays of objects and other unusual cases, but this rule is a good start.

reasonClassSizeLTE_C:

% The given class (associated with the constructor) is certain to be of this exact size.  The                                                                                                 
% reasoning is that we're able to track an allocation site with a known size to the constructor                                                                                               
% associated with the class.  There's a small bit of ambiguity about what the compiler will                                                                                                   
% generate for arrays of objects and other unusual cases, but this rule is a good start. 

So that's not great. We find two allocation sites for different sizes...

[sei-eschwartz]

[eschwartz@pd4 analysis]$ cat ../code/testcases/nfs_mill/NFS_Mill.exe.facts | fgrep -e sv_12071439086702097819 -e sv_1389056183026405998 | fgrep thisPtr
thisPtrAllocation(0x70e436, 0x70e410, sv_12071439086702097819, type_Heap, 0x14).
thisPtrAllocation(0x7fa366, 0x7fa340, sv_1389056183026405998, type_Heap, 0x20).
thisPtrUsage(0x70e450, 0x70e410, sv_12071439086702097819, 0x727290).
thisPtrUsage(0x7fa380, 0x7fa340, sv_1389056183026405998, 0x727290).
thisPtrOffset(sv_1389056183026405998, 0x14, sv_6071140477879548621).

I think what this is telling us is that there is some class that embeds/inherits a class at offset 0x14. This suggests there probably is a class of size 0x14. But also a class of size 0x20 that contains the class of size 0x14 at offset 0.

The above rules are flawed because they do not consider inheritance. If we call a constructor on an object allocated as size SZ, we know that the class size is at most SZ. But it may not be exactly SZ because it could include space for other base classes. If we know that there are no base classes, then it should be exactly SZ.

[sei-eschwartz]

I am testing a patch 🤞

[sei-eschwartz]

Seems to be working. Here is the branch if you want to want to try it out:

https://github.com/sei-eschwartz/pharos/tree/class_size_fix

[sei-eschwartz]

I was able to get a results and json file.

results.zip

There was a warning when producing the JSON:
Unable to find class 0xe45e4c. Please report this error to the OOAnalyzer developers.

I will look into that.

[sei-eschwartz]

@sei-ccohen We never resolved this

[sei-eschwartz]

Despite my comment above, I think this is resolved. Please reopen it if not.

[janbbeck]

The error does, in fact, seem to have been resolved. Thanks!