std.mtof(std.ftom(48)) => float foo; <<< foo >>>; foo $ int => int bar; <<< bar >>>; std.mtof(std.ftom(48)) $ int => int baz; <<< baz >>>; On OSX 10.3.9, using chuck 1.2.0.1, this results in: 48.000000 :(float) 47 :(int) 47 :(int) Cheers, - Graham
Hi Graham and all, I believe this is due to floating point (non)precision. 'foo' may actually have the value 47.99999999, and when printed out to 6 decimal places, is rounded up to 48 by the output formatting. However, the cast to int truncates that to 47. A dumb test (casting foo + .00000001): std.mtof(std.ftom(48)) => float foo; <<< foo >>>; (foo + .00000001) $ int => int bar; <<< bar >>>; results in: 48.000000 :(float) 48 :(int) It may be safer to round in many cases. For non-negegative values, either add .5 and cast the result to int, or call math.round and cast the result to int. For negative values, subtract .5 or use math.round. In testing this, we found that math.round (on win32 only) returns the incorrect result for negative numbers. This is now fixed in CVS and will be part of 1.2.0.2. Additionally, both std.mtof and std.ftom were previously implemented using single-precision, which would account for the (~.00000001) non-identity of mtof(ftom). I just changed both to use double precision, and now the original code all print 48 :(int). This will also be in 1.2.0.2. Thanks for finding this! Best, Ge! On Mon, 10 Oct 2005, Graham Percival wrote:
std.mtof(std.ftom(48)) => float foo; <<< foo >>>; foo $ int => int bar; <<< bar >>>; std.mtof(std.ftom(48)) $ int => int baz; <<< baz >>>;
On OSX 10.3.9, using chuck 1.2.0.1, this results in: 48.000000 :(float) 47 :(int) 47 :(int)
Cheers, - Graham
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
participants (2)
-
Ge Wang
-
Graham Percival