Because of Floating point arithmetic in Javascript, i understand 0.2 + 0.1 = 0.30000000000000004 will be in Javascript, but i don't understand why only multiply the figures by 10 i.e. let x = (0.2 * 10 + 0.1 * 10)…
For example, if I want to refactor the code : a-b>0 to a>b, are they identical? Without overflow, underflow and NaN, is there any a and b that a-b>0 and a>b results in different values? eg: would a-b and a>b…
For example, originally I have code like this: function(x,y){ let z=x+y; . . . } later I found y must be the same as x, and want to refactor x+y to x * 2, but need to ensure the behaviour…
My code is not working to convert 1.1 to a float. It works to convert 1.0, 1.25, and 1.5 to floats but doesn't convert 1.1. I do not understand why this would happen as they are all decimals with very…
I would like to understand the difference between Intl.NumberFormat and decimal.js library. if Intl.NumberFormat is capable of handling the currency calculations then where does decimal.js library stand and what benefit it does provider over Intl.NumberFormat This is specifically for Node.js…
I was viewing some code and found some parts use num/2, while others use num * 0.5. That inspired me a question : within the range of MIN_VALUE and MAX_VALUE, are num/2 and num*0.5 interchangeable? I was not only interested…
I've made a class that allows to split long double to an integer value and a binary exponent (for some precise calculations). My problem is pretty hard to reproduce because the class usually works great but on one specific machine…
I have mysql DB with important financial data, currently the data is stored as float type and I get incorrect data due to float rounding, I want to store it as DECIMAL. What is the safe way to convert the…
I have tested my code developed on a ubuntu 18.04 bionic docker image on a ubuntu 20.04 focal docker image. I saw that there were a problem with my unit test and I have narrowed the root cause to a…
I was wondering how Postgresql converts floating point (float4) values to NUMERIC. I chose 0.1 as a testing value. This value is not accurately representable in base2, see https://float.exposed/0x3dcccccd for a visualization. So the stored value 0x3dcccccd in hex for…