Silly decimal humans
Aug. 2nd, 2002 09:48 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dtmSo let's suppose you have a Java double (or float) and you want to come up with a string that represents the double/float in decimal but, and here's the kicker, you want the shortest string that'll adequately represent the number. For example, if you have the float corresponding to 0.85, when printing it out you might get:
0.8500000238418579
Not what you want (to get that in java you have to cast the float to a double, but you can get slightly less ugly artifacts simply by producing the float out of a long calculation - accumulated roundoff and whatnot).
What if you want to say "give me the shortest decimal assuming that this double's mantissa is only accurate to 20 bits"? (a java double has 23 bits in the mantissa)
Now suppose you want to do this in a tight loop where speed is critical. (otherwise, it wouldn't be much fun, would it?)
I've got vague references to stopping conditions using only integer arithmetic in this paper from 1992, but it's hard to clear though what the other stuff they say to get to what I want. Also, most of the paper assumes that the reader is fully aware of the contents of a presentation made at some ACM SIG* conference in 1990. Not too helpful, so I may just end up working it out on my own.
0.8500000238418579
Not what you want (to get that in java you have to cast the float to a double, but you can get slightly less ugly artifacts simply by producing the float out of a long calculation - accumulated roundoff and whatnot).
What if you want to say "give me the shortest decimal assuming that this double's mantissa is only accurate to 20 bits"? (a java double has 23 bits in the mantissa)
Now suppose you want to do this in a tight loop where speed is critical. (otherwise, it wouldn't be much fun, would it?)
I've got vague references to stopping conditions using only integer arithmetic in this paper from 1992, but it's hard to clear though what the other stuff they say to get to what I want. Also, most of the paper assumes that the reader is fully aware of the contents of a presentation made at some ACM SIG* conference in 1990. Not too helpful, so I may just end up working it out on my own.