All Questions

1971
votes
22answers
197389 views

Is floating point math broken?

0.1 + 0.2 == 0.3 -> false 0.1 + 0.2 -> 0.30000000000000004 Why does this happen?
189
votes
3answers
23406 views

Why are these numbers not equal?

The following code is obviously wrong. What's the problem? i <- 0.1 i <- i + 0.05 i ## [1] 0.15 if(i==0.15) cat("i equals 0.15") else cat("i does not equal 0.15") ## i does not equal 0.15
384
votes
34answers
230095 views

How to deal with floating point number precision in JavaScript?

I have the following dummy test script: function test(){ var x = 0.1 * 0.2; document.write(x); } test(); This will print the result 0.020000000000000004 while it should just print 0.02 (if ...
704
votes
14answers
198337 views

Why not use Double or Float to represent currency?

I've always been told never to represent money with double or float types, and this time I pose the question to you: why? I'm sure there is a very good reason, I simply do not know what it is.
127
votes
3answers
11987 views

Why Are Floating Point Numbers Inaccurate?

Why do some numbers lose accuracy when stored as floating point numbers? For example, the decimal number 9.2 can be expressed exactly as a ratio of two decimal integers (92/10), both of which can be ...
29
votes
7answers
76110 views

Floating point inaccuracy examples

How do you explain floating point inaccuracy to fresh programmers and laymen who still think computers are infinitely wise and accurate? Do you have a favourite example or anecdote which seems to get ...
52
votes
6answers
20054 views

Why is 24.0000 not equal to 24.0000 in MATLAB?

I am writing a program where I need to delete duplicate points stored in a matrix. The problem is that when it comes to check whether those points are in the matrix, MATLAB can't recognize them in the...
409
votes
25answers
284054 views

What is the most effective way for float and double comparison?

What would be the most efficient way to compare two double or two float values? Simply doing this is not correct: bool CompareDoubles1 (double A, double B) { return A == B; } But something like...
70
votes
7answers
127903 views

Division of integers in Java

This is a basic question but I can't find an answer. I've looked into floating point arithmetic and a few other topics but nothing has seemed to address this. I'm sure I just have the wrong terminol...
582
votes
13answers
479407 views

How can I force division to be floating point? Division keeps rounding down to 0

I have two integer values a and b, but I need their ratio in floating point. I know that a<b and I want to calculate a/b, so if I use integer division I'll always get 0 with a remainder of a. How...

Previous Next