Thanks Tyler. Floating point numbers are a problem in computer languages. Inaccurate calculations. When I was in the math program, we rounded numbers to the fifth decimal place.
I wonder why computer languages have such inaccuracy while calculators are accurate. Might it have something to do with binary calculations instead of base 10?
Thanks Tyler for answering Ebony's and my questions on the circuit board. Much appreciated.
Susanne Cardwell November 15, 2017 - 11:14am
The Landing is a social site for Athabasca University staff, students and invited guests. It is a space where they can share, communicate and connect with anyone or everyone.
Unless you are logged in, you will only be able to see the fraction of posts on the site that have been made public. Right now you are not logged in.
If you have an Athabasca University login ID, use your standard username and password to access this site.
We welcome comments on public posts from members of the public. Please note, however, that all comments made on public posts must be moderated by their owners before they become visible on the site. The owner of the post (and no one else) has to do that.
If you want the full range of features and you have a login ID, log in using the links at the top of the page or at https://landing.athabascau.ca/login (logins are secure and encrypted)
Posts made here are the responsibility of their owners and may not reflect the views of Athabasca University.
We block sites that track your web browsing without your permission. If a link is greyed out, click once to enable sharing, once more to share.
In fact, calculators have the same floating point limitations as computers, which makes sense, because calculators are simple computers. Try adding 0.1 to 1,000,000,000 and you'll get 1,000,000,000, not 1,000,000,000.1 (unless your calculator uses 64-bit numbers, which can precisely display up to about 18 digits); or try cos(0.00001) and you'll get 1, not 0.99999999995. They seem to be more accurate because they can limit their outputs. It's worth noting that having a hardware FPU does not guarantee greater precision than using a floating point library, as Arduinos do. The highest precision operations are actually done in software. Take a look at Java's BigDecimal for a great example and explanation.
You are correct in thinking that there could be a loss of precision when working with base-10 numbers in a binary system. The IEEE 754 standard defines 5 binary and 3 decimal floating point formats, where the latter can store decimal numbers exactly. Since the libm.a floating point library (Arduino/avr-gcc, etc.) likely uses the "binary32" format, it does not represent decimal number exactly, and is accurate to about 7 digits. Any format is no more precise than the value of the least significant bit ("LSB") (big-endian only; MSB value for little-endien) in the significand. For binary32 numbers, this is 2^-23 times the exponent, which is related to its magnitude. A quick search found a library that may be able to introduce the decimal32/64/128 formats to an Arduino platform, [here]: GitHub.com/toddtreece/esp8266-Arduino/.../decimal (it is for the ESP8266, a 32-bit Xtensa system, however, so it wouldn't work out of the box, if at all, on the 8-bit AVR/etc. systems we are using).
Tyler Lucas November 15, 2017 - 1:20pm