[chuck-users] int/float conversion oddity (48.0000 = 47)

[Ge Wang]

Hi Graham and all,

I believe this is due to floating point (non)precision.  'foo' may 
actually have the value 47.99999999, and when printed out to 6 decimal 
places, is rounded up to 48 by the output formatting.  However, the cast 
to int truncates that to 47.

A dumb test (casting foo + .00000001):

   std.mtof(std.ftom(48)) => float foo;
   <<< foo >>>;
   (foo + .00000001) $ int => int bar;
   <<< bar >>>;

results in:

   48.000000 :(float)
   48 :(int)


It may be safer to round in many cases.  For non-negegative values, 
either add .5 and cast the result to int, or call math.round and cast the 
result to int.  For negative values, subtract .5 or use math.round.

In testing this, we found that math.round (on win32 only) returns the 
incorrect result for negative numbers.  This is now fixed in CVS and will 
be part of 1.2.0.2.

Additionally, both std.mtof and std.ftom were previously implemented 
using single-precision, which would account for the (~.00000001) 
non-identity of mtof(ftom).  I just changed both to use double precision, 
and now the original code all print 48 :(int).  This will also be in 
1.2.0.2.

Thanks for finding this!

Best,
Ge!



On Mon, 10 Oct 2005, Graham Percival wrote:

> std.mtof(std.ftom(48)) => float foo;
> <<< foo >>>;
> foo $ int => int bar;
> <<< bar >>>;
> std.mtof(std.ftom(48)) $ int => int baz;
> <<< baz >>>;
>
>
> On OSX 10.3.9, using chuck 1.2.0.1, this results in:
> 48.000000 :(float)
> 47 :(int)
> 47 :(int)
>
>
> Cheers,
> - Graham
>
> _______________________________________________
> chuck-users mailing list
> chuck-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
>