Ad space (banner)
๐Ÿ”ทVB.NET Lessons
Lesson 57 / 68

Watch out for floating-point error

This lesson covers floating-point error in VB.NET, with the aim of understanding why decimal arithmetic can give surprising results.

Computers hold decimals in binary, so even tidy-looking values such as 0.1 and 0.2 cannot be represented exactly, and a small error creeps in.

The sample code shows that although 0.1 + 0.2 looks as though it should be 0.3, the internal error makes the comparison 0.1 + 0.2 = 0.3 come out false. This happens in most languages that use floating point.

A common early stumble is comparing decimals directly with = and not getting the expected result. Comparing decimals calls for allowing for the error — checking that the difference between the two is small enough.

In professional work, calculations that cannot tolerate error — money, for instance — are better handled with a higher-precision type such as Decimal.

๐Ÿ“– Reference code
โœ๏ธ Your code
Type your code, then press "Run"

๐Ÿงช This site can't compile or run VB.NET directly, so it checks on the spot whether what you typed matches the reference code (scoring happens entirely in your browser โ€” nothing is sent anywhere).

Ad space (banner)
Ad space (in-article)