[Oberon] Bug in Texts.WriteReal of PO

[Hans Klaver]

Hi all,

Recently I encountered a (minor) bug in the output of the current version of Texts.WriteReal: 
- if parameter n (the number of characters for the output string) is 
  less then 7 the rounding is not correct
- if parameter n = 7 the output is way off mark
- otherwise (n > 7) the output is correct

E.g. when the number 1554.70E5 is input and then written with different values for n we get the following output:

 n  Texts.WriteReal
--------------------
 0  1.E+08
 1  1.E+08
 2  1.E+08
 3  1.E+08
 4  1.E+08
 5  1.E+08
 7  6.E+08
 8  2.E+08
 9  1.6E+08
10  1.55E+08
11  1.555E+08
12  1.5547E+08
13  1.55470E+08
14  1.554700E+08
15   1.554700E+08

Above for n < 9 only the line with n = 8  2.E+08 is correct.

I found two alternative fixes:
- Fix A has as welcome (imho) side effect that it also ensures at least one digit after the decimal point for n < 9; this is the output of the Oberon-90 version of Texts.WriteReal (by Jürg Gutknecht).
- Fix B only corrects the rounding for n < 7 and the erroneous output in case n = 7.

The module below shows the corrected procedure WriteReal with the two fixes in place. I prefer fix A because I think floating point output without at least one decimal digit is pointless. 

I also took the opportunity to propose a better tabulation of the case x = 0.0


MODULE Test;
  IMPORT In, Fonts, Texts, Oberon;
  VAR W: Texts.Writer; 

  PROCEDURE Ten(n: INTEGER): REAL;
    VAR t, p: REAL;
  BEGIN t := 1.0; p := 10.0;   (*compute 10^n *)
    WHILE n > 0 DO
      IF ODD(n) THEN t := p * t END ;
      p := p*p; n := n DIV 2
    END ;
    RETURN t
  END Ten;

  PROCEDURE WriteReal* (VAR W: Texts.Writer; x: REAL; n: INTEGER);  (* hk  2 Oct 2020 *)
    CONST minLen = 7;  maxLen = 14;  (* excl. one leading space and sign *)
    VAR e, i, k, m: INTEGER;
      d: ARRAY 16 OF CHAR;
  BEGIN e := ASR(ORD(x), 23) MOD 100H;  (*binary exponent*)
    IF e = 0 THEN
      WHILE n > maxLen DO Texts.Write(W, " "); DEC(n) END;              (* tabulate 0 *)               
      Texts.WriteString(W, "  0      ");                                                                
      WHILE n > (minLen + 2) DO Texts.Write(W, " "); DEC(n) END                 
    ELSIF e = 255 THEN Texts.WriteString(W, " NaN ")
    ELSE 
      IF n < 9 THEN n := 9 END;     (* fix A, also >= 1 digit after the decimal point *)
      Texts.Write(W, " ");
      WHILE n >= 15 DO DEC(n); Texts.Write(W, " ") END ;
      (* 2 < n < 9 digits to be written*)
      IF x < 0.0 THEN Texts.Write(W, "-"); x := -x ELSE Texts.Write(W, " ") END ;
      e := (e - 127) * 77 DIV 256 - 6;  (*decimal exponent*)
      IF e >= 0 THEN x := x / Ten(e) ELSE x := Ten(-e) * x END ;
      m := FLOOR(x + 0.5);   
      IF m >= 10000000 THEN INC(e); m := m DIV 10 END; 
      i := 0; k := 13-n; 
  (*  IF k > 5 THEN k := 5 END;   *)                                         (* fix B *)
      REPEAT
        IF i = k THEN INC(m, 5) END ;    (*rounding*)
        d[i] := CHR(m MOD 10 + 30H); m := m DIV 10; INC(i)
      UNTIL m = 0;
      DEC(i); Texts.Write(W, d[i]); Texts.Write(W, ".");
      IF i < n-7 THEN n := 0 ELSE n := 14 - n END ;
      WHILE i > n DO DEC(i); Texts.Write(W, d[i]) END ;
      Texts.Write(W, "E"); INC(e, 6);
      IF e < 0 THEN Texts.Write(W, "-"); e := -e ELSE Texts.Write(W, "+") END ;
      Texts.Write(W, CHR(e DIV 10 + 30H)); Texts.Write(W, CHR(e MOD 10 + 30H))
    END
  END WriteReal;

  PROCEDURE Do*;
    VAR x: REAL; n, k: INTEGER;
  BEGIN
    In.Open;  In.Real(x);  In.Int(k);
    IF In.Done THEN
      Texts.SetFont(W, Fonts.This("Courier10.Fnt"));
      FOR n := 0 TO 15 DO   
        Texts.WriteInt(W, n, 2);  Texts.WriteReal(W, x, n);  WriteReal(W, x, n);  
        Texts.WriteRealFix(W, x, n, k);  Texts.WriteLn(W)
      END
    ELSE 
      Texts.WriteString(W, "Parameter error")
    END;
    Texts.WriteLn(W);  Texts.Append(Oberon.Log, W.buf)
  END Do;

BEGIN Texts.OpenWriter(W)
END Test.Do^

1554.70E5  1
1554.70E5  2


Run Test.Do with the (start of) the first line selected to see the effect of the fix.

If you then run the command with the second line selected you can see another quirk of floating point output in module Texts, in this case in Texts.WriteRealFix, but the same may happen with Texts.WriteReal: integer overflow is not trapped, so you get wrong output; in this case the digits of a well known Mersenne prime, but in other instances less helpful real numbers. 

On that I will write a separate post.

--
Hans Klaver