overflowing 32-bit integer in ML

[pwxy]

Hello Ray,

I noticed that in your August 26 17:15:44 check-in for ml_MultiLevelPreconditioner_Smoothers.cpp, you changed the following two lines from:

double local[2];
double global[2];

to:

int local[2];
int global[2];

What was the reason for this change?

Isn't this a bug? Because global[1] stores the "# estim. global nnz =", and for our MHD problems with 8 PDEs per mesh node, for greater than 10 million row matrices we will overflow a 32-bit int (since we will have greater than 2 billion global nnz). I was going to change them back to doubles, but I realized I should check with you first.

thanks
-paul

[pwxy]

Hi Ray,

64-bit integers would work too (as an alternative to double). I think Jonathan had switched them from "int" to "double" ~10 years ago to avoid the overflow. I'll let you push the changes to the repo since it affects some of your other code.

thanks
-paul

[rstumin]

Having some problems compile on cee-compute011 today. Not sure what it is ... it looks like something in the openmpi
changed late last week ... and this seems to be affecting my builds. I've got a couple of other meetings ... but will
try to get to this soon.

-Ray

On 09/06/16 08:16, pwxy wrote:

Hi Ray,

64-bit integers would work too (as an alternative to double). I think Jonathan had switched them from "int" to "double"
~10 years ago to avoid the overflow. I'll let you push the changes to the repo since it affects some of your other code.

thanks
-paul

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
#598 (comment), or mute the thread
https://github.com/notifications/unsubscribe-auth/ALR0uT1n3VyUqMrdNv6lw6f4aagJV_LSks5qnYO6gaJpZM4J124Q.

Ray Tuminaro phone: (925) 294-2564
MS 9159 fax: (925) 294-2234
Sandia National Laboratories email: rstumin@sandia.gov
PO Box 969 http://www.cs.sandia.gov/~rstumin
Livermore, CA 94551

[aprokop]

This was fixed in 4072573.
The fix was:

-    int local[2];
-    int global[2];
+    long int local[2];
+    long int global[2];

[mhoemmen]

Not really a fix, since long int can be 32 bits on some platforms. long long is guaranteed to be >= 64 bits, but requires C99/C++11.

[aprokop]

True. But since the last communication was in September, and nobody complained since, and there was a patch dedicated to this...