Extend the test of MonsterExtra (#5428)

* Extend the test of MonsterExtra

- Extend C++ test of MonsterExtra
- Add conversion of fbs/json NaNs to unsigned quiet-NaN
- Update documentation (cross-platform interoperability)

* Fix declaration of infinity constants int the test

docs/source/CppUsage.md

@@ -35,7 +35,7 @@ The test code itself is located in
 [test.cpp](https://github.com/google/flatbuffers/blob/master/tests/test.cpp).
 
 This test file is built alongside `flatc`. To review how to build the project,
-please read the [Building](@ref flatbuffers_guide_building) documenation.
+please read the [Building](@ref flatbuffers_guide_building) documentation.
 
 To run the tests, execute `flattests` from the root `flatbuffers/` directory.
 For example, on [Linux](https://en.wikipedia.org/wiki/Linux), you would simply
@@ -546,21 +546,63 @@ locale-independent or locale-narrow functions `strtof_l`, `strtod_l`,
 These functions use specified locale rather than the global or per-thread
 locale instead. They are part of POSIX-2008 but not part of the C/C++
 standard library, therefore, may be missing on some platforms.
-
 The Flatbuffers library try to detect these functions at configuration and
 compile time:
-- `_MSC_VER >= 1900`: check MSVC2012 or higher for MSVC buid
-- `_XOPEN_SOURCE>=700`: check POSIX-2008 for GCC/Clang build
-- `check_cxx_symbol_exists(strtof_l stdlib.h)`: CMake check of `strtod_f`
+- CMake `"CMakeLists.txt"`:
+  - Check existence of `strtol_l` and `strtod_l` in the `<stdlib.h>`.
+- Compile-time `"/include/base.h"`:
+  - `_MSC_VER >= 1900`: MSVC2012 or higher if build with MSVC.
+  - `_XOPEN_SOURCE>=700`: POSIX-2008 if build with GCC/Clang.
 
 After detection, the definition `FLATBUFFERS_LOCALE_INDEPENDENT` will be
 set to `0` or `1`.
+To override or stop this detection use CMake `-DFLATBUFFERS_LOCALE_INDEPENDENT={0|1}`
+or predefine `FLATBUFFERS_LOCALE_INDEPENDENT` symbol.
 
-It is possible to test the compatibility of the Flatbuffers library with
-a specific locale using the environment variable `FLATBUFFERS_TEST_LOCALE`:
+To test the compatibility of the Flatbuffers library with
+a specific locale use the environment variable `FLATBUFFERS_TEST_LOCALE`:
 ```sh
 >FLATBUFFERS_TEST_LOCALE="" ./flattests
 >FLATBUFFERS_TEST_LOCALE="ru_RU.CP1251" ./flattests
 ```
 
+## Support of floating-point numbers
+The Flatbuffers library assumes that a C++ compiler and a CPU are 
+compatible with the `IEEE-754` floating-point standard.
+The schema and json parser may fail if `fast-math` or `/fp:fast` mode is active.
+
+### Support of hexadecimal and special floating-point numbers
+According to the [grammar](@ref flatbuffers_grammar) `fbs` and `json` files 
+may use hexadecimal and special (`NaN`, `Inf`) floating-point literals.
+The Flatbuffers uses `strtof` and `strtod` functions to parse floating-point
+literals. The Flatbuffers library has a code to detect a compiler compatibility 
+with the literals. If necessary conditions are met the preprocessor constant
+`FLATBUFFERS_HAS_NEW_STRTOD` will be set to `1`.
+The support of floating-point literals will be limited at compile time
+if `FLATBUFFERS_HAS_NEW_STRTOD` constant is less than `1`.
+In this case, schemas with hexadecimal or special literals cannot be used.
+
+### Comparison of floating-point NaN values
+The floating-point `NaN` (`not a number`) is special value which 
+representing an undefined or unrepresentable value.
+`NaN` may be explicitly assigned to variables, typically as a representation 
+for missing values or may be a result of a mathematical operation.
+The `IEEE-754` defines two kind of `NaNs`:
+- Quiet NaNs, or `qNaNs`.
+- Signaling NaNs, or `sNaNs`.
+
+According to the `IEEE-754`, a comparison with `NaN` always returns 
+an unordered result even when compared with itself. As a result, a whole 
+Flatbuffers object will be not equal to itself if has one or more `NaN`.
+Flatbuffers scalar fields that have the default value are not actually stored 
+in the serialized data but are generated in code (see [Writing a schema](@ref flatbuffers_guide_writing_schema)).
+Scalar fields with `NaN` defaults break this behavior.
+If a schema has a lot of `NaN` defaults the Flatbuffers can override 
+the unordered comparison by the ordered: `(NaN==NaN)->true`.
+This ordered comparison is enabled when compiling a program with the symbol
+`FLATBUFFERS_NAN_DEFAULTS` defined.
+Additional computations added by `FLATBUFFERS_NAN_DEFAULTS` are very cheap
+if GCC or Clang used. These compilers have a compile-time implementation 
+of `isnan` checking which MSVC does not.
+
 <br>

docs/source/Internals.md

@@ -15,6 +15,12 @@ all commonly used CPUs today. FlatBuffers will also work on big-endian
 machines, but will be slightly slower because of additional
 byte-swap intrinsics.
 
+It is assumed that the following conditions are met, to ensure
+cross-platform interoperability:
+- The binary `IEEE-754` format is used for floating-point numbers.
+- The `two's complemented` representation is used for signed integers.
+- The endianness is the same for floating-point numbers as for integers.
+
 On purpose, the format leaves a lot of details about where exactly
 things live in memory undefined, e.g. fields in a table can have any
 order, and objects to some extent can be stored in many orders. This is

docs/source/Schemas.md

@@ -439,14 +439,19 @@ numerical literals:
     For example: `[0x123, +0x45, -0x67]` are equal to `[291, 69, -103]` decimals.
 -   The format of float-point numbers is fully compatible with C/C++ format.
     If a modern C++ compiler is used the parser accepts hexadecimal and special
-    float-point literals as well:
+    floating-point literals as well:
     `[-1.0, 2., .3e0, 3.e4, 0x21.34p-5, -inf, nan]`.
-    The exponent suffix of hexadecimal float-point number is mandatory.
 
-    Extended float-point support was tested with:
+    The following conventions for floating-point numbers are used:
+    - The exponent suffix of hexadecimal floating-point number is mandatory.
+    - Parsed `NaN` converted to unsigned IEEE-754 `quiet-NaN` value.
+
+    Extended floating-point support was tested with:
     - x64 Windows: `MSVC2015` and higher.
     - x64 Linux: `LLVM 6.0`, `GCC 4.9` and higher.
 
+    For details, see [Use in C++](@ref flatbuffers_guide_use_cpp) section.
+
 -   For compatibility with a JSON lint tool all numeric literals of scalar
     fields can be wrapped to quoted string:
     `"1", "2.0", "0x48A", "0x0C.0Ep-1", "-inf", "true"`.

include/flatbuffers/flatbuffers.h

@@ -25,16 +25,21 @@
 
 namespace flatbuffers {
 // Generic 'operator==' with conditional specialisations.
+// T e - new value of a scalar field.
+// T def - default of scalar (is known at compile-time).
 template<typename T> inline bool IsTheSameAs(T e, T def) { return e == def; }
 
 #if defined(FLATBUFFERS_NAN_DEFAULTS) && \
-    (!defined(_MSC_VER) || _MSC_VER >= 1800)
+    defined(FLATBUFFERS_HAS_NEW_STRTOD) && (FLATBUFFERS_HAS_NEW_STRTOD > 0)
 // Like `operator==(e, def)` with weak NaN if T=(float|double).
+template<typename T> inline bool IsFloatTheSameAs(T e, T def) {
+  return (e == def) || ((def != def) && (e != e));
+}
 template<> inline bool IsTheSameAs<float>(float e, float def) {
-  return (e == def) || (std::isnan(def) && std::isnan(e));
+  return IsFloatTheSameAs(e, def);
 }
 template<> inline bool IsTheSameAs<double>(double e, double def) {
-  return (e == def) || (std::isnan(def) && std::isnan(e));
+  return IsFloatTheSameAs(e, def);
 }
 #endif
 

src/idl_parser.cpp

@@ -19,7 +19,7 @@
 #include <string>
 #include <utility>
 
-#include <math.h>
+#include <cmath>
 
 #include "flatbuffers/idl.h"
 #include "flatbuffers/util.h"
@@ -1525,6 +1525,21 @@ CheckedError Parser::TokenError() {
   return Error("cannot parse value starting with: " + TokenToStringId(token_));
 }
 
+// Re-pack helper (ParseSingleValue) to normalize defaults of scalars.
+template<typename T> inline void SingleValueRepack(Value &e, T val) {
+  // Remove leading zeros.
+  if (IsInteger(e.type.base_type)) { e.constant = NumToString(val); }
+}
+#if defined(FLATBUFFERS_HAS_NEW_STRTOD) && (FLATBUFFERS_HAS_NEW_STRTOD > 0)
+// Normilaze defaults NaN to unsigned quiet-NaN(0).
+static inline void SingleValueRepack(Value& e, float val) {
+  if (val != val) e.constant = "nan";
+}
+static inline void SingleValueRepack(Value& e, double val) {
+  if (val != val) e.constant = "nan";
+}
+#endif
+
 CheckedError Parser::ParseSingleValue(const std::string *name, Value &e,
                                       bool check_now) {
   // First see if this could be a conversion function:
@@ -1569,96 +1584,94 @@ CheckedError Parser::ParseSingleValue(const std::string *name, Value &e,
   }
 
   auto match = false;
+  const auto in_type = e.type.base_type;
   // clang-format off
-  #define TRY_ECHECK(force, dtoken, check, req)    \
+  #define IF_ECHECK_(force, dtoken, check, req)    \
     if (!match && ((check) || IsConstTrue(force))) \
     ECHECK(TryTypedValue(name, dtoken, check, e, req, &match))
+  #define TRY_ECHECK(dtoken, check, req) IF_ECHECK_(false, dtoken, check, req)
+  #define FORCE_ECHECK(dtoken, check, req) IF_ECHECK_(true, dtoken, check, req)
   // clang-format on
 
   if (token_ == kTokenStringConstant || token_ == kTokenIdentifier) {
     const auto kTokenStringOrIdent = token_;
     // The string type is a most probable type, check it first.
-    TRY_ECHECK(false, kTokenStringConstant,
-               e.type.base_type == BASE_TYPE_STRING, BASE_TYPE_STRING);
+    TRY_ECHECK(kTokenStringConstant, in_type == BASE_TYPE_STRING,
+               BASE_TYPE_STRING);
 
     // avoid escaped and non-ascii in the string
-    if ((token_ == kTokenStringConstant) && IsScalar(e.type.base_type) &&
+    if (!match && (token_ == kTokenStringConstant) && IsScalar(in_type) &&
         !attr_is_trivial_ascii_string_) {
       return Error(
           std::string("type mismatch or invalid value, an initializer of "
                       "non-string field must be trivial ASCII string: type: ") +
-          kTypeNames[e.type.base_type] + ", name: " + (name ? *name : "") +
+          kTypeNames[in_type] + ", name: " + (name ? *name : "") +
           ", value: " + attribute_);
     }
 
     // A boolean as true/false. Boolean as Integer check below.
-    if (!match && IsBool(e.type.base_type)) {
+    if (!match && IsBool(in_type)) {
       auto is_true = attribute_ == "true";
       if (is_true || attribute_ == "false") {
         attribute_ = is_true ? "1" : "0";
         // accepts both kTokenStringConstant and kTokenIdentifier
-        TRY_ECHECK(false, kTokenStringOrIdent, IsBool(e.type.base_type),
-                   BASE_TYPE_BOOL);
+        TRY_ECHECK(kTokenStringOrIdent, IsBool(in_type), BASE_TYPE_BOOL);
       }
     }
     // Check if this could be a string/identifier enum value.
     // Enum can have only true integer base type.
-    if (!match && IsInteger(e.type.base_type) && !IsBool(e.type.base_type) &&
+    if (!match && IsInteger(in_type) && !IsBool(in_type) &&
         IsIdentifierStart(*attribute_.c_str())) {
       ECHECK(ParseEnumFromString(e.type, &e.constant));
       NEXT();
       match = true;
     }
-    // float/integer number in string
-    if ((token_ == kTokenStringConstant) && IsScalar(e.type.base_type)) {
+    // Parse a float/integer number from the string.
+    if (!match) check_now = true;  // Re-pack if parsed from string literal.
+    if (!match && (token_ == kTokenStringConstant) && IsScalar(in_type)) {
       // remove trailing whitespaces from attribute_
       auto last = attribute_.find_last_not_of(' ');
       if (std::string::npos != last)  // has non-whitespace
         attribute_.resize(last + 1);
     }
     // Float numbers or nan, inf, pi, etc.
-    TRY_ECHECK(false, kTokenStringOrIdent, IsFloat(e.type.base_type),
-               BASE_TYPE_FLOAT);
+    TRY_ECHECK(kTokenStringOrIdent, IsFloat(in_type), BASE_TYPE_FLOAT);
     // An integer constant in string.
-    TRY_ECHECK(false, kTokenStringOrIdent, IsInteger(e.type.base_type),
-               BASE_TYPE_INT);
+    TRY_ECHECK(kTokenStringOrIdent, IsInteger(in_type), BASE_TYPE_INT);
     // Unknown tokens will be interpreted as string type.
-    TRY_ECHECK(true, kTokenStringConstant, e.type.base_type == BASE_TYPE_STRING,
+    FORCE_ECHECK(kTokenStringConstant, in_type == BASE_TYPE_STRING,
                BASE_TYPE_STRING);
   } else {
     // Try a float number.
-    TRY_ECHECK(false, kTokenFloatConstant, IsFloat(e.type.base_type),
-               BASE_TYPE_FLOAT);
+    TRY_ECHECK(kTokenFloatConstant, IsFloat(in_type), BASE_TYPE_FLOAT);
     // Integer token can init any scalar (integer of float).
-    TRY_ECHECK(true, kTokenIntegerConstant, IsScalar(e.type.base_type),
-               BASE_TYPE_INT);
+    FORCE_ECHECK(kTokenIntegerConstant, IsScalar(in_type), BASE_TYPE_INT);
   }
-  #undef TRY_ECHECK
+#undef FORCE_ECHECK
+#undef TRY_ECHECK
+#undef IF_ECHECK_
 
   if (!match) return TokenError();
-
+  const auto match_type = e.type.base_type; // may differ from in_type
   // The check_now flag must be true when parse a fbs-schema.
   // This flag forces to check default scalar values or metadata of field.
   // For JSON parser the flag should be false.
   // If it is set for JSON each value will be checked twice (see ParseTable).
-  if (check_now && IsScalar(e.type.base_type)) {
-    // "re-pack" an integer scalar to remove any ambiguities like leading zeros
-    // which can be treated as octal-literal (idl_gen_cpp/GenDefaultConstant).
-    const auto repack = IsInteger(e.type.base_type);
-    switch (e.type.base_type) {
+  if (check_now && IsScalar(match_type)) {
     // clang-format off
+    switch (match_type) {
     #define FLATBUFFERS_TD(ENUM, IDLTYPE, \
             CTYPE, JTYPE, GTYPE, NTYPE, PTYPE, RTYPE) \
             case BASE_TYPE_ ## ENUM: {\
                 CTYPE val; \
                 ECHECK(atot(e.constant.c_str(), *this, &val)); \
-                if(repack) e.constant = NumToString(val); \
+                SingleValueRepack(e, val); \
               break; }
     FLATBUFFERS_GEN_TYPES_SCALAR(FLATBUFFERS_TD);
     #undef FLATBUFFERS_TD
     default: break;
-    // clang-format on
     }
+    // clang-format on
   }
   return NoError();
 }
@@ -3031,7 +3044,7 @@ static Namespace *GetNamespace(
     for (;;) {
       dot = qualified_name.find('.', pos);
       if (dot == std::string::npos) { break; }
-      ns->components.push_back(qualified_name.substr(pos, dot-pos));
+      ns->components.push_back(qualified_name.substr(pos, dot - pos));
       pos = dot + 1;
     }
   }

tests/MyGame/MonsterExtra.py

@@ -19,100 +19,116 @@ class MonsterExtra(object):
         self._tab = flatbuffers.table.Table(buf, pos)
 
     # MonsterExtra
-    def TestfNan(self):
+    def D0(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(4))
         if o != 0:
-            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
+            return self._tab.Get(flatbuffers.number_types.Float64Flags, o + self._tab.Pos)
         return float('nan')
 
     # MonsterExtra
-    def TestfPinf(self):
+    def D1(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(6))
         if o != 0:
-            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
-        return float('inf')
+            return self._tab.Get(flatbuffers.number_types.Float64Flags, o + self._tab.Pos)
+        return float('nan')
 
     # MonsterExtra
-    def TestfNinf(self):
+    def D2(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(8))
         if o != 0:
-            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
-        return float('-inf')
+            return self._tab.Get(flatbuffers.number_types.Float64Flags, o + self._tab.Pos)
+        return float('inf')
 
     # MonsterExtra
-    def TestdNan(self):
+    def D3(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(10))
         if o != 0:
             return self._tab.Get(flatbuffers.number_types.Float64Flags, o + self._tab.Pos)
-        return float('nan')
+        return float('-inf')
 
     # MonsterExtra
-    def TestdPinf(self):
+    def F0(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(12))
         if o != 0:
-            return self._tab.Get(flatbuffers.number_types.Float64Flags, o + self._tab.Pos)
-        return float('inf')
+            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
+        return float('nan')
 
     # MonsterExtra
-    def TestdNinf(self):
+    def F1(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(14))
         if o != 0:
-            return self._tab.Get(flatbuffers.number_types.Float64Flags, o + self._tab.Pos)
-        return float('-inf')
+            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
+        return float('nan')
 
     # MonsterExtra
-    def TestfVec(self, j):
+    def F2(self):
         o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(16))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
+        return float('inf')
+
+    # MonsterExtra
+    def F3(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(18))
+        if o != 0:
+            return self._tab.Get(flatbuffers.number_types.Float32Flags, o + self._tab.Pos)
+        return float('-inf')
+
+    # MonsterExtra
+    def Dvec(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(20))
         if o != 0:
             a = self._tab.Vector(o)
-            return self._tab.Get(flatbuffers.number_types.Float32Flags, a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 4))
+            return self._tab.Get(flatbuffers.number_types.Float64Flags, a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 8))
         return 0
 
     # MonsterExtra
-    def TestfVecAsNumpy(self):
-        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(16))
+    def DvecAsNumpy(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(20))
         if o != 0:
-            return self._tab.GetVectorAsNumpy(flatbuffers.number_types.Float32Flags, o)
+            return self._tab.GetVectorAsNumpy(flatbuffers.number_types.Float64Flags, o)
         return 0
 
     # MonsterExtra
-    def TestfVecLength(self):
-        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(16))
+    def DvecLength(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(20))
         if o != 0:
             return self._tab.VectorLen(o)
         return 0
 
     # MonsterExtra
-    def TestdVec(self, j):
-        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(18))
+    def Fvec(self, j):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(22))
         if o != 0:
             a = self._tab.Vector(o)
-            return self._tab.Get(flatbuffers.number_types.Float64Flags, a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 8))
+            return self._tab.Get(flatbuffers.number_types.Float32Flags, a + flatbuffers.number_types.UOffsetTFlags.py_type(j * 4))
         return 0
 
     # MonsterExtra
-    def TestdVecAsNumpy(self):
-        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(18))
+    def FvecAsNumpy(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(22))
         if o != 0:
-            return self._tab.GetVectorAsNumpy(flatbuffers.number_types.Float64Flags, o)
+            return self._tab.GetVectorAsNumpy(flatbuffers.number_types.Float32Flags, o)
         return 0
 
     # MonsterExtra
-    def TestdVecLength(self):
-        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(18))
+    def FvecLength(self):
+        o = flatbuffers.number_types.UOffsetTFlags.py_type(self._tab.Offset(22))
         if o != 0:
             return self._tab.VectorLen(o)
         return 0
 
-def MonsterExtraStart(builder): builder.StartObject(8)
-def MonsterExtraAddTestfNan(builder, testfNan): builder.PrependFloat32Slot(0, testfNan, float('nan'))
-def MonsterExtraAddTestfPinf(builder, testfPinf): builder.PrependFloat32Slot(1, testfPinf, float('inf'))
-def MonsterExtraAddTestfNinf(builder, testfNinf): builder.PrependFloat32Slot(2, testfNinf, float('-inf'))
-def MonsterExtraAddTestdNan(builder, testdNan): builder.PrependFloat64Slot(3, testdNan, float('nan'))
-def MonsterExtraAddTestdPinf(builder, testdPinf): builder.PrependFloat64Slot(4, testdPinf, float('inf'))
-def MonsterExtraAddTestdNinf(builder, testdNinf): builder.PrependFloat64Slot(5, testdNinf, float('-inf'))
-def MonsterExtraAddTestfVec(builder, testfVec): builder.PrependUOffsetTRelativeSlot(6, flatbuffers.number_types.UOffsetTFlags.py_type(testfVec), 0)
-def MonsterExtraStartTestfVecVector(builder, numElems): return builder.StartVector(4, numElems, 4)
-def MonsterExtraAddTestdVec(builder, testdVec): builder.PrependUOffsetTRelativeSlot(7, flatbuffers.number_types.UOffsetTFlags.py_type(testdVec), 0)
-def MonsterExtraStartTestdVecVector(builder, numElems): return builder.StartVector(8, numElems, 8)
+def MonsterExtraStart(builder): builder.StartObject(10)
+def MonsterExtraAddD0(builder, d0): builder.PrependFloat64Slot(0, d0, float('nan'))
+def MonsterExtraAddD1(builder, d1): builder.PrependFloat64Slot(1, d1, float('nan'))
+def MonsterExtraAddD2(builder, d2): builder.PrependFloat64Slot(2, d2, float('inf'))
+def MonsterExtraAddD3(builder, d3): builder.PrependFloat64Slot(3, d3, float('-inf'))
+def MonsterExtraAddF0(builder, f0): builder.PrependFloat32Slot(4, f0, float('nan'))
+def MonsterExtraAddF1(builder, f1): builder.PrependFloat32Slot(5, f1, float('nan'))
+def MonsterExtraAddF2(builder, f2): builder.PrependFloat32Slot(6, f2, float('inf'))
+def MonsterExtraAddF3(builder, f3): builder.PrependFloat32Slot(7, f3, float('-inf'))
+def MonsterExtraAddDvec(builder, dvec): builder.PrependUOffsetTRelativeSlot(8, flatbuffers.number_types.UOffsetTFlags.py_type(dvec), 0)
+def MonsterExtraStartDvecVector(builder, numElems): return builder.StartVector(8, numElems, 8)
+def MonsterExtraAddFvec(builder, fvec): builder.PrependUOffsetTRelativeSlot(9, flatbuffers.number_types.UOffsetTFlags.py_type(fvec), 0)
+def MonsterExtraStartFvecVector(builder, numElems): return builder.StartVector(4, numElems, 4)
 def MonsterExtraEnd(builder): return builder.EndObject()

tests/monster_extra.fbs

@@ -3,14 +3,16 @@ namespace MyGame;
 // Not all programming languages support this extra table.
 table MonsterExtra {
   // Float-point values with NaN and Inf defaults.
-  testf_nan:float = nan;
-  testf_pinf:float = +inf;
-  testf_ninf:float = -inf;
-  testd_nan:double = nan;
-  testd_pinf:double = +inf;
-  testd_ninf:double = -inf;
-  testf_vec : [float];
-  testd_vec : [double];
+  d0:double = nan;
+  d1:double = -nan; // parser must ignore sign of NaN
+  d2:double = +inf;
+  d3:double = -inf;
+  f0:float = -nan; // parser must ignore sign of NaN
+  f1:float = +nan;
+  f2:float = +inf;
+  f3:float = -inf;
+  dvec : [double];
+  fvec : [float];
 }
 
 root_type MonsterExtra;

tests/monsterdata_extra.json

 {
-  // Float-point values with NaN and Inf defaults.
-  testf_nan : nan,
-  testf_pinf : +inf,
-  testf_ninf : -inf,
-  testd_nan  : nan,
-  testd_pinf : +inf,
-  testd_ninf : -inf,
-  testf_vec : [-1.0, 2.0, -inf, +inf, nan],
-  testd_vec : [-1.0, 4.0, -inf, +inf, nan]
+  // Initialize with non-default values.
+  d0 : -nan, // match with default
+  d1 : +inf,
+  d2 : -inf,
+  d3:   nan,
+  f0 : +nan, // match with default
+  f1 : -nan, // match with default
+  f2 : +inf, // match with default
+  f3 : -inf, // match with default
+  // Values should have exact binary representation 
+  // to avoid rounding effects in tests.
+  dvec : [2.0, +inf, -inf, nan,],
+  fvec : [1.0, -inf, +inf, nan],
 }

tests/py_test.py

@@ -1412,13 +1412,13 @@ class TestAllCodePathsOfMonsterExtraSchema(unittest.TestCase):
         self.mon = MyGame.MonsterExtra.MonsterExtra.GetRootAsMonsterExtra(b.Bytes, b.Head())
 
     def test_default_nan_inf(self):
-        self.assertTrue(math.isnan(self.mon.TestfNan()))
-        self.assertEqual(self.mon.TestfPinf(), float("inf"))
-        self.assertEqual(self.mon.TestfNinf(), float("-inf"))
+        self.assertTrue(math.isnan(self.mon.F1()))
+        self.assertEqual(self.mon.F2(), float("inf"))
+        self.assertEqual(self.mon.F3(), float("-inf"))
 
-        self.assertTrue(math.isnan(self.mon.TestdNan()))
-        self.assertEqual(self.mon.TestdPinf(), float("inf"))
-        self.assertEqual(self.mon.TestdNinf(), float("-inf"))
+        self.assertTrue(math.isnan(self.mon.D1()))
+        self.assertEqual(self.mon.D2(), float("inf"))
+        self.assertEqual(self.mon.D3(), float("-inf"))
 
 
 class TestVtableDeduplication(unittest.TestCase):

tests/test.cpp

@@ -41,6 +41,25 @@
 
 #include "flatbuffers/flexbuffers.h"
 
+// clang-format off
+// Check that char* and uint8_t* are interoperable types.
+// The reinterpret_cast<> between the pointers are used to simplify data loading.
+static_assert(flatbuffers::is_same<uint8_t, char>::value ||
+              flatbuffers::is_same<uint8_t, unsigned char>::value,
+              "unexpected uint8_t type");
+
+#if defined(FLATBUFFERS_HAS_NEW_STRTOD) && (FLATBUFFERS_HAS_NEW_STRTOD > 0)
+  // Ensure IEEE-754 support if tests of floats with NaN/Inf will run.
+  static_assert(std::numeric_limits<float>::is_iec559 &&
+                std::numeric_limits<double>::is_iec559,
+                "IEC-559 (IEEE-754) standard required");
+#endif
+// clang-format on
+
+// Shortcuts for the infinity.
+static const auto infinityf = std::numeric_limits<float>::infinity();
+static const auto infinityd = std::numeric_limits<double>::infinity();
+
 using namespace MyGame::Example;
 
 void FlatBufferBuilderTest();
@@ -601,36 +620,105 @@ void JsonDefaultTest() {
   TEST_EQ(std::string::npos != jsongen.find("testf: 3.14159"), true);
 }
 
-#if defined(FLATBUFFERS_HAS_NEW_STRTOD)
+#if defined(FLATBUFFERS_HAS_NEW_STRTOD) && (FLATBUFFERS_HAS_NEW_STRTOD > 0)
+// The IEEE-754 quiet_NaN is not simple binary constant.
+// All binary NaN bit strings have all the bits of the biased exponent field E
+// set to 1. A quiet NaN bit string should be encoded with the first bit d[1]
+// of the trailing significand field T being 1 (d[0] is implicit bit).
+// It is assumed that endianness of floating-point is same as integer.
+template<typename T, typename U, U qnan_base> bool is_quiet_nan_impl(T v) {
+  static_assert(sizeof(T) == sizeof(U), "unexpected");
+  U b = 0;
+  std::memcpy(&b, &v, sizeof(T));
+  return ((b & qnan_base) == qnan_base);
+}
+static bool is_quiet_nan(float v) {
+  return is_quiet_nan_impl<float, uint32_t, 0x7FC00000u>(v);
+}
+static bool is_quiet_nan(double v) {
+  return is_quiet_nan_impl<double, uint64_t, 0x7FF8000000000000ul>(v);
+}
+
 void TestMonsterExtraFloats() {
+  TEST_EQ(is_quiet_nan(1.0), false);
+  TEST_EQ(is_quiet_nan(infinityd), false);
+  TEST_EQ(is_quiet_nan(-infinityf), false);
+  TEST_EQ(is_quiet_nan(std::numeric_limits<float>::quiet_NaN()), true);
+  TEST_EQ(is_quiet_nan(std::numeric_limits<double>::quiet_NaN()), true);
+
+  using namespace flatbuffers;
   using namespace MyGame;
   // Load FlatBuffer schema (.fbs) from disk.
   std::string schemafile;
-  TEST_EQ(flatbuffers::LoadFile((test_data_path + "monster_extra.fbs").c_str(),
-                                false, &schemafile),
+  TEST_EQ(LoadFile((test_data_path + "monster_extra.fbs").c_str(), false,
+                   &schemafile),
           true);
   // Parse schema first, so we can use it to parse the data after.
-  flatbuffers::Parser parser;
-  auto include_test_path =
-      flatbuffers::ConCatPathFileName(test_data_path, "include_test");
+  Parser parser;
+  auto include_test_path = ConCatPathFileName(test_data_path, "include_test");
   const char *include_directories[] = { test_data_path.c_str(),
                                         include_test_path.c_str(), nullptr };
   TEST_EQ(parser.Parse(schemafile.c_str(), include_directories), true);
   // Create empty extra and store to json.
   parser.opts.output_default_scalars_in_json = true;
   parser.opts.output_enum_identifiers = true;
-  flatbuffers::FlatBufferBuilder builder;
-  MonsterExtraBuilder extra(builder);
-  FinishMonsterExtraBuffer(builder, extra.Finish());
+  FlatBufferBuilder builder;
+  const auto def_root = MonsterExtraBuilder(builder).Finish();
+  FinishMonsterExtraBuffer(builder, def_root);
+  const auto def_obj = builder.GetBufferPointer();
+  const auto def_extra = GetMonsterExtra(def_obj);
+  TEST_NOTNULL(def_extra);
+  TEST_EQ(is_quiet_nan(def_extra->f0()), true);
+  TEST_EQ(is_quiet_nan(def_extra->f1()), true);
+  TEST_EQ(def_extra->f2(), +infinityf);
+  TEST_EQ(def_extra->f3(), -infinityf);
+  TEST_EQ(is_quiet_nan(def_extra->d0()), true);
+  TEST_EQ(is_quiet_nan(def_extra->d1()), true);
+  TEST_EQ(def_extra->d2(), +infinityd);
+  TEST_EQ(def_extra->d3(), -infinityd);
   std::string jsongen;
-  auto result = GenerateText(parser, builder.GetBufferPointer(), &jsongen);
+  auto result = GenerateText(parser, def_obj, &jsongen);
   TEST_EQ(result, true);
-  TEST_EQ(std::string::npos != jsongen.find("testf_nan: nan"), true);
-  TEST_EQ(std::string::npos != jsongen.find("testf_pinf: inf"), true);
-  TEST_EQ(std::string::npos != jsongen.find("testf_ninf: -inf"), true);
-  TEST_EQ(std::string::npos != jsongen.find("testd_nan: nan"), true);
-  TEST_EQ(std::string::npos != jsongen.find("testd_pinf: inf"), true);
-  TEST_EQ(std::string::npos != jsongen.find("testd_ninf: -inf"), true);
+  // Check expected default values.
+  TEST_EQ(std::string::npos != jsongen.find("f0: nan"), true);
+  TEST_EQ(std::string::npos != jsongen.find("f1: nan"), true);
+  TEST_EQ(std::string::npos != jsongen.find("f2: inf"), true);
+  TEST_EQ(std::string::npos != jsongen.find("f3: -inf"), true);
+  TEST_EQ(std::string::npos != jsongen.find("d0: nan"), true);
+  TEST_EQ(std::string::npos != jsongen.find("d1: nan"), true);
+  TEST_EQ(std::string::npos != jsongen.find("d2: inf"), true);
+  TEST_EQ(std::string::npos != jsongen.find("d3: -inf"), true);
+  // Parse 'mosterdata_extra.json'.
+  const auto extra_base = test_data_path + "monsterdata_extra";
+  jsongen = "";
+  TEST_EQ(LoadFile((extra_base + ".json").c_str(), false, &jsongen), true);
+  TEST_EQ(parser.Parse(jsongen.c_str()), true);
+  const auto test_file = parser.builder_.GetBufferPointer();
+  const auto test_size = parser.builder_.GetSize();
+  Verifier verifier(test_file, test_size);
+  TEST_ASSERT(VerifyMonsterExtraBuffer(verifier));
+  const auto extra = GetMonsterExtra(test_file);
+  TEST_NOTNULL(extra);
+  TEST_EQ(is_quiet_nan(extra->f0()), true);
+  TEST_EQ(is_quiet_nan(extra->f1()), true);
+  TEST_EQ(extra->f2(), +infinityf);
+  TEST_EQ(extra->f3(), -infinityf);
+  TEST_EQ(is_quiet_nan(extra->d0()), true);
+  TEST_EQ(extra->d1(), +infinityd);
+  TEST_EQ(extra->d2(), -infinityd);
+  TEST_EQ(is_quiet_nan(extra->d3()), true);
+  TEST_NOTNULL(extra->fvec());
+  TEST_EQ(extra->fvec()->size(), 4);
+  TEST_EQ(extra->fvec()->Get(0), 1.0f);
+  TEST_EQ(extra->fvec()->Get(1), -infinityf);
+  TEST_EQ(extra->fvec()->Get(2), +infinityf);
+  TEST_EQ(is_quiet_nan(extra->fvec()->Get(3)), true);
+  TEST_NOTNULL(extra->dvec());
+  TEST_EQ(extra->dvec()->size(), 4);
+  TEST_EQ(extra->dvec()->Get(0), 2.0);
+  TEST_EQ(extra->dvec()->Get(1), +infinityd);
+  TEST_EQ(extra->dvec()->Get(2), -infinityd);
+  TEST_EQ(is_quiet_nan(extra->dvec()->Get(3)), true);
 }
 #else
 void TestMonsterExtraFloats() {}
@@ -1663,8 +1751,6 @@ void IntegerBoundaryTest() {
 }
 
 void ValidFloatTest() {
-  const auto infinityf = flatbuffers::numeric_limits<float>::infinity();
-  const auto infinityd = flatbuffers::numeric_limits<double>::infinity();
   // check rounding to infinity
   TEST_EQ(TestValue<float>("{ Y:+3.4029e+38 }", "float"), +infinityf);
   TEST_EQ(TestValue<float>("{ Y:-3.4029e+38 }", "float"), -infinityf);
@@ -1694,7 +1780,7 @@ void ValidFloatTest() {
   TEST_EQ(TestValue<float>("{ Y:5 }", "float"), 5.0f);
   TEST_EQ(TestValue<float>("{ Y:\"5\" }", "float"), 5.0f);
 
-  #if defined(FLATBUFFERS_HAS_NEW_STRTOD)
+  #if defined(FLATBUFFERS_HAS_NEW_STRTOD) && (FLATBUFFERS_HAS_NEW_STRTOD > 0)
   // Old MSVC versions may have problem with this check.
   // https://www.exploringbinary.com/visual-c-plus-plus-strtod-still-broken/
   TEST_EQ(TestValue<double>("{ Y:6.9294956446009195e15 }", "double"),
@@ -1740,7 +1826,7 @@ void ValidFloatTest() {
 
 #else   // FLATBUFFERS_HAS_NEW_STRTOD
   TEST_OUTPUT_LINE("FLATBUFFERS_HAS_NEW_STRTOD tests skipped");
-#endif  // FLATBUFFERS_HAS_NEW_STRTOD
+#endif  // !FLATBUFFERS_HAS_NEW_STRTOD
 }
 
 void InvalidFloatTest() {