[Oberon] real math

Sean Case seancase at tpg.com.au
Thu Nov 13 13:48:20 CET 2014


On 13 Nov 2014, at 23:28, oberon at moravcik.info wrote:

>   float number;

‘number’ is a single precision variable.

>   number = 480.0 / 100.0 * 100.0;

I believe that this expression is evaluated in double precision
before being assigned to the single precision variable. This has the
effect of rounding off the errors.

The errors arise in the first place because the computer represents
floating point numbers as binary fractions, as opposed to the decimal
fractions used by most calculators. 4.8 as a binary fraction is a
recurring sequence, like, say, 1/3 considered as a decimal fraction.

Sean Case





More information about the Oberon mailing list