Home > Web Front-end > JS Tutorial > Example analysis of float operation precision in javascript_javascript skills

Example analysis of float operation precision in javascript_javascript skills

WBOY
Release: 2016-05-16 18:20:49
Original
1268 people have browsed it

Someone asked a js question:

Copy code The code is as follows:

var i = 0.07;
var r = i*100;
alert(r);

Why is the result 7.0000000000000001?
After checking the information, we actually know that in JavsScript, when variables are stored There is no distinction between number and float types, but they are stored uniformly as float. JavaScript uses the 64-bit floating point format defined by the IEEE 754-2008 standard to store numbers. According to the definition of IEEE 754: http://en.wikipedia.org/wiki/IEEE_754-2008
The length of the integer corresponding to decimal64 is 10, The length of the decimal part is 16, so the default calculation result is "7.0000000000000001". If the last decimal is 0, take 1 as the valid digit flag.

Similarly, we can imagine that the result of 1/3 should be 0.3333333333333333.
So how to correct this value?
You can use the following methods:
1. parseInt

var r4=parseInt(i*100);

2. Math.round

var r2=Math.round((i*100)*1000)/1000;

Both of the above two methods can get 7
Attached is the full test code:
Copy code The code is as follows:



Test script





Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template