Fork me on GitHub
#test-check
<
2018-04-20
>
devn19:04:00

If i want to generate a double with limited precision, how do?

gfredericks20:04:46

Expressed in base 10 or base 2?

gfredericks20:04:46

And do you mean precision relative to the fixed decimal point, or in the floating point sense?

gfredericks20:04:09

(like is 1e-100 high precision?)

devn20:04:33

yes, relative to the fixed decimal point

gfredericks20:04:01

I'd generate an integer and divide it by 100.0 probably

gfredericks20:04:52

Obligatory warning about how doubles can't precisely represent most of those numbers