All Questions

1971
votes
22answers
197389 views

Is floating point math broken?

0.1 + 0.2 == 0.3 -> false 0.1 + 0.2 -> 0.30000000000000004 Why does this happen?
189
votes
3answers
23406 views

Why are these numbers not equal?

The following code is obviously wrong. What's the problem? i <- 0.1 i <- i + 0.05 i ## [1] 0.15 if(i==0.15) cat("i equals 0.15") else cat("i does not equal 0.15") ## i does not equal 0.15
29
votes
7answers
76110 views

Floating point inaccuracy examples

How do you explain floating point inaccuracy to fresh programmers and laymen who still think computers are infinitely wise and accurate? Do you have a favourite example or anecdote which seems to get ...
355
votes
9answers
78190 views

How dangerous is it to compare floating point values?

I know UIKit uses CGFloat because of the resolution independent coordinate system. But every time I want to check if for example frame.origin.x is 0 it makes me feel sick: if (theView.frame.origin.x...
67
votes
9answers
29089 views

PHP - Floating Number Precision

$a = '35'; $b = '-34.99'; echo ($a + $b); Results in 0.009999999999998 What is up with that? I wondered why my program kept reporting odd results. Why doesn't PHP return the expected 0.01?
142
votes
21answers
123962 views

What's wrong with using == to compare floats in Java?

According to this java.sun page == is the equality comparison operator for floating point numbers in Java. However, when I type this code: if(sectionID == currentSectionID) into my editor and run ...
36
votes
6answers
21990 views

Large numbers erroneously rounded in Javascript

See this code: <html> <head> <script src="http://www.json.org/json2.js" type="text/javascript"></script> <script type="text/javascript"> var jsonString = '{"id":7...
42
votes
2answers
34069 views

What is the standard solution in JavaScript for handling big numbers (BigNum)?

Is there a bignum library for JavaScript or built-in that I can include like <script type="text/javascript" src="the_bignum_library.js"></script> ? I think my users would prefer to ent...
26
votes
11answers
38213 views

How to avoid floating point precision errors with floats or doubles in Java?

I have a very annoying problem with long sums of floats or doubles in Java. Basically the idea is that if I execute: for ( float value = 0.0f; value < 1.0f; value += 0.1f ) System.out.println(...
58
votes
5answers
107557 views

Dealing with float precision in Javascript

I have a large amount of numeric values y in javascript. I want to group them by rounding them down to the nearest multiple of x and convert the result to a string. How do I get around the annoying...

Previous Next