Bug #320
Budget inconsistencies
Start date:
02/20/2015
Due date:
% Done:
0%
Description
when comparing a run with 45 procs with one with 9 procs (both with reduced grid, only latitudinal mpi decomposition), output (mmix, restart) are identical bit-for-bit, except the budgets:
check budini ... 28432 (out of 39168) differences found in budini max diff. in : [8, 21, 33] (rel.diff. 0.000000 %) where field #1 = 6.9922226493487276e+28 where field #2 = 6.9922226493487382e+28 check budadvx ... 38668 (out of 39168) differences found in budadvx max diff. in : [8, 21, 33] (rel.diff. 3500.000000 %) where field #1 = 7123649972633.0312 where field #2 = 256451399014789.22 check budadvy ... 38516 (out of 39168) differences found in budadvy max diff. in : [8, 21, 33] (rel.diff. -0.000000 %) where field #1 = -2.1878826532546108e+27 where field #2 = -2.1878826532549399e+27
while the 0% are expected (for aggregated budgets, the different mpi layout change the order of operations), the 3500% is puzzling and requires explanation.
History
#1 Updated by Philippe Le Sager over 7 years ago
- Subject changed from X-advection budget to Budget inconsistencies
Further comparisons between 45 and 22 cores, still for a short 3h-run. I isolate three buggy budgets:
check budadvx ... 38320 (out of 39168) differences found in budadvx max diff. in : [8, 20, 33] (rel.diff. -114.423077 %) where field #1 = -61738299762819.633 where field #2 = 8904562465791.291 check budchem ... 44700 (out of 50796) differences found in budchem max diff. in : [9, 22, 33] (rel.diff. 164.438222 %) where field #1 = 766920097792.0 where field #2 = -494189674496.0 check budconv_cp ... 30714 (out of 39168) differences found in budconv_cp max diff. in : [1, 5, 14] (rel.diff. -6.499508 %) where field #1 = -2.2055309866594325e+22 where field #2 = -2.3488796472526816e+22
Again, j-stat, mmix and restart files are bit-for-bit identical: this is purely a budget issue.
#2 Updated by Philippe Le Sager over 6 years ago
- Priority changed from Normal to Low