Merge pull request #13909 from kinchungwong:logging_20190220

OE-11 Logging revamp (#13909)

* Initial commit for log tag support.

Part of #11003, incomplete. Should pass build.

Moved LogLevel enum to logger.defines.hpp

LogTag struct used to convey both name and log level threshold as
one argument to the new logging macro. See logtag.hpp file, and
CV_LOG_WITH_TAG macro.

Global log level is now associated with a global log tag, when a
logging statement doesn't specify any log tag. See getLogLevel and
getGlobalLogTag functions.

A macro CV_LOGTAG_FALLBACK is allowed to be re-defined by other modules
or compilation units, internally, so that logging statements inside
that unit that specify NULL as tag will fall back to the re-defined tag.

Line-of-code information (file name, line number, function name),
together with tag name, are passed into the new log message sink.
See writeLogMessageEx function.

Fixed old incorrect CV_LOG_VERBOSE usage in ocl4dnn_conv_spatial.cpp.

* Implemented tag-based log filtering

Added LogTagManager. This is an initial version, using standard C++
approach as much as possible, to allow easier code review. Will
optimize later.

A workaround for all static dynamic initialization issues is
implemented. Refer to code comments.

* Added LogTagConfigParser.

Note: new code does not fully handle old log config parsing behavior.

* Fix log tag config vs registering ordering issue.

* Started testing LogTagConfigParser, incomplete.

The intention of this commit is to illustrate the capabilities of
the current design of LogTagConfigParser.

The test contained in this commit is not complete. Also, design changes
may require throwing away this commit and rewriting test code from
scratch.

Does not test whitespace segmentation (multiple tags on the config);
will do in next commit.

* Added CV_LOGTAG_EXPAND_NAME macro

This macro allows to be re-defined locally in other compilation units
to apply a prefix to whatever argument is passed as the "tag" argument
into CV_LOG_WITH_TAG. The default definition in logger.hpp does not
modify the argument. It is recommended to include the address-of
operator (ampersand) when re-defined locally.

* Added a few tests for LogTagManager, some fail.

See test_logtagmanager.cpp
Failed tests are: non-global ("something"), setting level by name-part
(first part or any part) has no effect at all.

* LogTagManagerTests substring non-confusion tests

* Fix major bugs in LogTagManager

The code change is intended to approximate the spec documented in
https://gist.github.com/kinchungwong/ec25bc1eba99142e0be4509b0f67d0c6

Refer to test suite in test_logtagmanager.cpp

Filter test result in "opencv_test_core" ...
with gtest_filter "LogTagManager*"

To see the test code that finds the bugs, refer to original commits
(before rebase; might be gone)

.. f3451208 (2019-03-03T19:45:17Z)
.... LogTagManagerTests substring non-confusion tests

.. 1b848f5f (2019-03-03T01:55:18Z)
.... Added a few tests for LogTagManager, some fail.

* Added LogTagManagerNamePartNonConfusionTest.

See test_logtagmanager.cpp in modules/core/test.

* Added LogTagAuto for auto registration in ctor

* Rewritten LogTagManager to resolve issues.

* Resolves code review issues around 2019-04-10

LogTagConfigParser::parseLogLevel - as part of resolving code review
issues, this function is rewritten to simplify control flow and to
improve conformance with legacy usage (for string values "OFF",
"DISABLED", and "WARNINGS").

modules/core/include/opencv2/core/utils/logger.defines.hpp

@@ -17,6 +17,26 @@
 #define CV_LOG_LEVEL_DEBUG 5           //!< Debug message. Disabled in the "Release" build.
 #define CV_LOG_LEVEL_VERBOSE 6         //!< Verbose (trace) messages. Requires verbosity level. Disabled in the "Release" build.
 
+namespace cv {
+namespace utils {
+namespace logging {
+
+//! Supported logging levels and their semantic
+enum LogLevel {
+    LOG_LEVEL_SILENT = 0,              //!< for using in setLogVevel() call
+    LOG_LEVEL_FATAL = 1,               //!< Fatal (critical) error (unrecoverable internal error)
+    LOG_LEVEL_ERROR = 2,               //!< Error message
+    LOG_LEVEL_WARNING = 3,             //!< Warning message
+    LOG_LEVEL_INFO = 4,                //!< Info message
+    LOG_LEVEL_DEBUG = 5,               //!< Debug message. Disabled in the "Release" build.
+    LOG_LEVEL_VERBOSE = 6,             //!< Verbose (trace) messages. Requires verbosity level. Disabled in the "Release" build.
+#ifndef CV_DOXYGEN
+    ENUM_LOG_LEVEL_FORCE_INT = INT_MAX
+#endif
+};
+
+}}} // namespace
+
 //! @}
 
 #endif // OPENCV_LOGGER_DEFINES_HPP

modules/core/include/opencv2/core/utils/logger.hpp

@@ -10,6 +10,7 @@
 #include <limits.h> // INT_MAX
 
 #include "logger.defines.hpp"
+#include "logtag.hpp"
 
 //! @addtogroup core_logging
 // This section describes OpenCV logging utilities.
@@ -20,20 +21,6 @@ namespace cv {
 namespace utils {
 namespace logging {
 
-//! Supported logging levels and their semantic
-enum LogLevel {
-    LOG_LEVEL_SILENT = 0,              //!< for using in setLogVevel() call
-    LOG_LEVEL_FATAL = 1,               //!< Fatal (critical) error (unrecoverable internal error)
-    LOG_LEVEL_ERROR = 2,               //!< Error message
-    LOG_LEVEL_WARNING = 3,             //!< Warning message
-    LOG_LEVEL_INFO = 4,                //!< Info message
-    LOG_LEVEL_DEBUG = 5,               //!< Debug message. Disabled in the "Release" build.
-    LOG_LEVEL_VERBOSE = 6,             //!< Verbose (trace) messages. Requires verbosity level. Disabled in the "Release" build.
-#ifndef CV_DOXYGEN
-    ENUM_LOG_LEVEL_FORCE_INT = INT_MAX
-#endif
-};
-
 /** Set global logging level
 @return previous logging level
 */
@@ -41,15 +28,39 @@ CV_EXPORTS LogLevel setLogLevel(LogLevel logLevel);
 /** Get global logging level */
 CV_EXPORTS LogLevel getLogLevel();
 
+CV_EXPORTS void registerLogTag(cv::utils::logging::LogTag* plogtag);
+
+CV_EXPORTS void setLogTagLevel(const char* tag, cv::utils::logging::LogLevel level);
+
+CV_EXPORTS cv::utils::logging::LogLevel getLogTagLevel(const char* tag);
+
 namespace internal {
+
+/** Get global log tag */
+CV_EXPORTS cv::utils::logging::LogTag* getGlobalLogTag();
+
 /** Write log message */
 CV_EXPORTS void writeLogMessage(LogLevel logLevel, const char* message);
+
+/** Write log message */
+CV_EXPORTS void writeLogMessageEx(LogLevel logLevel, const char* tag, const char* file, int line, const char* func, const char* message);
+
 } // namespace
 
+struct LogTagAuto
+    : public LogTag
+{
+    inline LogTagAuto(const char* _name, LogLevel _level)
+        : LogTag(_name, _level)
+    {
+        registerLogTag(this);
+    }
+};
+
 /**
  * \def CV_LOG_STRIP_LEVEL
  *
- * Define CV_LOG_STRIP_LEVEL=CV_LOG_LEVEL_[DEBUG|INFO|WARN|ERROR|FATAL|DISABLED] to compile out anything at that and before that logging level
+ * Define CV_LOG_STRIP_LEVEL=CV_LOG_LEVEL_[DEBUG|INFO|WARN|ERROR|FATAL|SILENT] to compile out anything at that and before that logging level
  */
 #ifndef CV_LOG_STRIP_LEVEL
 # if defined NDEBUG
@@ -59,27 +70,84 @@ CV_EXPORTS void writeLogMessage(LogLevel logLevel, const char* message);
 # endif
 #endif
 
+#define CV_LOGTAG_PTR_CAST(expr) static_cast<const cv::utils::logging::LogTag*>(expr)
+
+// CV_LOGTAG_EXPAND_NAME is intended to be re-defined (undef and then define again)
+// to allows logging users to use a shorter name argument when calling
+// CV_LOG_WITH_TAG or its related macros such as CV_LOG_INFO.
+//
+// This macro is intended to modify the tag argument as a string (token), via
+// preprocessor token pasting or metaprogramming techniques. A typical usage
+// is to apply a prefix, such as
+// ...... #define CV_LOGTAG_EXPAND_NAME(tag) cv_logtag_##tag
+//
+// It is permitted to re-define to a hard-coded expression, ignoring the tag.
+// This would work identically like the CV_LOGTAG_FALLBACK macro.
+//
+// Important: When the logging macro is called with tag being NULL, a user-defined
+// CV_LOGTAG_EXPAND_NAME may expand it into cv_logtag_0, cv_logtag_NULL, or
+// cv_logtag_nullptr. Use with care. Also be mindful of C++ symbol redefinitions.
+//
+// If there is significant amount of logging code with tag being NULL, it is
+// recommended to use (re-define) CV_LOGTAG_FALLBACK to inject locally a default
+// tag at the beginning of a compilation unit, to minimize lines of code changes.
+//
+#define CV_LOGTAG_EXPAND_NAME(tag) tag
+
+// CV_LOGTAG_FALLBACK is intended to be re-defined (undef and then define again)
+// by any other compilation units to provide a log tag when the logging statement
+// does not specify one. The macro needs to expand into a C++ expression that can
+// be static_cast into (cv::utils::logging::LogTag*). Null (nullptr) is permitted.
+#define CV_LOGTAG_FALLBACK nullptr
+
+// CV_LOGTAG_GLOBAL is the tag used when a log tag is not specified in the logging
+// statement nor the compilation unit. The macro needs to expand into a C++
+// expression that can be static_cast into (cv::utils::logging::LogTag*). Must be
+// non-null. Do not re-define.
+#define CV_LOGTAG_GLOBAL cv::utils::logging::internal::getGlobalLogTag()
+
+#define CV_LOG_WITH_TAG(tag, msgLevel, ...) \
+    for(;;) { \
+        const auto cv_temp_msglevel = (cv::utils::logging::LogLevel)(msgLevel); \
+        if (cv_temp_msglevel >= (CV_LOG_STRIP_LEVEL)) break; \
+        auto cv_temp_logtagptr = CV_LOGTAG_PTR_CAST(CV_LOGTAG_EXPAND_NAME(tag)); \
+        if (!cv_temp_logtagptr) cv_temp_logtagptr = CV_LOGTAG_PTR_CAST(CV_LOGTAG_FALLBACK); \
+        if (!cv_temp_logtagptr) cv_temp_logtagptr = CV_LOGTAG_PTR_CAST(CV_LOGTAG_GLOBAL); \
+        if (cv_temp_logtagptr && (cv_temp_msglevel > cv_temp_logtagptr->level)) break; \
+        std::stringstream cv_temp_logstream; \
+        cv_temp_logstream << __VA_ARGS__; \
+        cv::utils::logging::internal::writeLogMessageEx( \
+            cv_temp_msglevel, \
+            (cv_temp_logtagptr ? cv_temp_logtagptr->name : nullptr), \
+            __FILE__, \
+            __LINE__, \
+            CV_Func, \
+            cv_temp_logstream.str().c_str()); \
+        break; \
+    }
+
+#define CV_LOG_FATAL(tag, ...)   CV_LOG_WITH_TAG(tag, cv::utils::logging::LOG_LEVEL_FATAL, __VA_ARGS__)
+#define CV_LOG_ERROR(tag, ...)   CV_LOG_WITH_TAG(tag, cv::utils::logging::LOG_LEVEL_ERROR, __VA_ARGS__)
+#define CV_LOG_WARNING(tag, ...) CV_LOG_WITH_TAG(tag, cv::utils::logging::LOG_LEVEL_WARNING, __VA_ARGS__)
+#define CV_LOG_INFO(tag, ...) CV_LOG_WITH_TAG(tag, cv::utils::logging::LOG_LEVEL_INFO, __VA_ARGS__)
+#define CV_LOG_DEBUG(tag, ...) CV_LOG_WITH_TAG(tag, cv::utils::logging::LOG_LEVEL_DEBUG, __VA_ARGS__)
+#define CV_LOG_VERBOSE(tag, v, ...) CV_LOG_WITH_TAG(tag, (cv::utils::logging::LOG_LEVEL_VERBOSE + (int)(v)), __VA_ARGS__)
 
-#define CV_LOG_FATAL(tag, ...)   for(;;) { if (cv::utils::logging::getLogLevel() < cv::utils::logging::LOG_LEVEL_FATAL) break; std::stringstream ss; ss << __VA_ARGS__; cv::utils::logging::internal::writeLogMessage(cv::utils::logging::LOG_LEVEL_FATAL, ss.str().c_str()); break; }
-#define CV_LOG_ERROR(tag, ...)   for(;;) { if (cv::utils::logging::getLogLevel() < cv::utils::logging::LOG_LEVEL_ERROR) break; std::stringstream ss; ss << __VA_ARGS__; cv::utils::logging::internal::writeLogMessage(cv::utils::logging::LOG_LEVEL_ERROR, ss.str().c_str()); break; }
-#define CV_LOG_WARNING(tag, ...) for(;;) { if (cv::utils::logging::getLogLevel() < cv::utils::logging::LOG_LEVEL_WARNING) break; std::stringstream ss; ss << __VA_ARGS__; cv::utils::logging::internal::writeLogMessage(cv::utils::logging::LOG_LEVEL_WARNING, ss.str().c_str()); break; }
 #if CV_LOG_STRIP_LEVEL <= CV_LOG_LEVEL_INFO
-#define CV_LOG_INFO(tag, ...)
-#else
-#define CV_LOG_INFO(tag, ...)    for(;;) { if (cv::utils::logging::getLogLevel() < cv::utils::logging::LOG_LEVEL_INFO) break; std::stringstream ss; ss << __VA_ARGS__; cv::utils::logging::internal::writeLogMessage(cv::utils::logging::LOG_LEVEL_INFO, ss.str().c_str()); break; }
+# undef CV_LOG_INFO
+# define CV_LOG_INFO(tag, ...)
 #endif
+
 #if CV_LOG_STRIP_LEVEL <= CV_LOG_LEVEL_DEBUG
-#define CV_LOG_DEBUG(tag, ...)
-#else
-#define CV_LOG_DEBUG(tag, ...)   for(;;) { if (cv::utils::logging::getLogLevel() < cv::utils::logging::LOG_LEVEL_DEBUG) break; std::stringstream ss; ss << __VA_ARGS__; cv::utils::logging::internal::writeLogMessage(cv::utils::logging::LOG_LEVEL_DEBUG, ss.str().c_str()); break; }
+# undef CV_LOG_DEBUG
+# define CV_LOG_DEBUG(tag, ...)
 #endif
+
 #if CV_LOG_STRIP_LEVEL <= CV_LOG_LEVEL_VERBOSE
-#define CV_LOG_VERBOSE(tag, v, ...)
-#else
-#define CV_LOG_VERBOSE(tag, v, ...) for(;;) { if (cv::utils::logging::getLogLevel() < cv::utils::logging::LOG_LEVEL_VERBOSE) break; std::stringstream ss; ss << "[VERB" << v << ":" << cv::utils::getThreadID() << "] " << __VA_ARGS__; cv::utils::logging::internal::writeLogMessage(cv::utils::logging::LOG_LEVEL_VERBOSE, ss.str().c_str()); break; }
+# undef CV_LOG_VERBOSE
+# define CV_LOG_VERBOSE(tag, v, ...)
 #endif
 
-
 }}} // namespace
 
 //! @}

modules/core/include/opencv2/core/utils/logtag.hpp

+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef OPENCV_CORE_LOGTAG_HPP
+#define OPENCV_CORE_LOGTAG_HPP
+
+#include "opencv2/core/cvstd.hpp"
+#include "logger.defines.hpp"
+
+namespace cv {
+namespace utils {
+namespace logging {
+
+struct LogTag
+{
+    const char* name;
+    LogLevel level;
+
+    inline constexpr LogTag(const char* _name, LogLevel _level)
+        : name(_name)
+        , level(_level)
+    {}
+};
+
+}}}
+
+#endif

modules/core/src/logger.cpp

@@ -6,6 +6,8 @@
 
 #include <opencv2/core/utils/configuration.private.hpp>
 #include <opencv2/core/utils/logger.hpp>
+#include "utils/logtagmanager.hpp"
+#include "utils/logtagconfigparser.hpp"
 
 #include <sstream>
 #include <iostream>
@@ -19,52 +21,156 @@ namespace cv {
 namespace utils {
 namespace logging {
 
-static LogLevel parseLogLevelConfiguration()
+namespace internal
 {
-    static cv::String param_log_level = utils::getConfigurationParameterString("OPENCV_LOG_LEVEL",
+
+// Combining several things that require static dynamic initialization in a
+// well-defined order into a struct.
+//
+struct GlobalLoggingInitStruct
+{
+public:
 #if defined NDEBUG
-            "WARNING"
+    static constexpr bool m_isDebugBuild = false;
 #else
-            "INFO"
+    static constexpr bool m_isDebugBuild = true;
 #endif
-    );
-    if (param_log_level == "DISABLED" || param_log_level == "disabled" ||
-        param_log_level == "0" || param_log_level == "OFF" || param_log_level == "off")
-        return LOG_LEVEL_SILENT;
-    if (param_log_level == "FATAL" || param_log_level == "fatal")
-        return LOG_LEVEL_FATAL;
-    if (param_log_level == "ERROR" || param_log_level == "error")
-        return LOG_LEVEL_ERROR;
-    if (param_log_level == "WARNING" || param_log_level == "warning" ||
-        param_log_level == "WARNINGS" || param_log_level == "warnings" ||
-        param_log_level == "WARN" || param_log_level == "warn")
-        return LOG_LEVEL_WARNING;
-    if (param_log_level == "INFO" || param_log_level == "info")
-        return LOG_LEVEL_INFO;
-    if (param_log_level == "DEBUG" || param_log_level == "debug")
-        return LOG_LEVEL_DEBUG;
-    if (param_log_level == "VERBOSE" || param_log_level == "verbose")
-        return LOG_LEVEL_VERBOSE;
-    std::cerr << "ERROR: Unexpected logging level value: " << param_log_level << std::endl;
-    return LOG_LEVEL_INFO;
+
+public:
+    static constexpr LogLevel m_defaultUnconfiguredGlobalLevel =
+        m_isDebugBuild ? LOG_LEVEL_DEBUG : LOG_LEVEL_WARNING;
+
+public:
+    LogTagManager logTagManager;
+
+    GlobalLoggingInitStruct()
+        : logTagManager(m_defaultUnconfiguredGlobalLevel)
+    {
+        applyConfigString();
+        handleMalformed();
+    }
+
+private:
+    void applyConfigString()
+    {
+        logTagManager.setConfigString(utils::getConfigurationParameterString("OPENCV_LOG_LEVEL", ""));
+    }
+
+    void handleMalformed()
+    {
+        // need to print warning for malformed log tag config strings?
+        if (m_isDebugBuild)
+        {
+            const auto& parser = logTagManager.getConfigParser();
+            if (parser.hasMalformed())
+            {
+                const auto& malformedList = parser.getMalformed();
+                for (const auto& malformed : malformedList)
+                {
+                    std::cout << "Malformed log level config: \"" << malformed << "\"\n";
+                }
+                std::cout.flush();
+            }
+        }
+    }
+};
+
+// Static dynamic initialization guard function for the combined struct
+// just defined above
+//
+// An initialization guard function guarantees that outside code cannot
+// accidentally see not-yet-dynamically-initialized data, by routing
+// all outside access request to this function, so that this function
+// has a chance to run the initialization code if necessary.
+//
+// An initialization guard function only guarantees initialization upon
+// the first call to this function.
+//
+static GlobalLoggingInitStruct& getGlobalLoggingInitStruct()
+{
+    static GlobalLoggingInitStruct globalLoggingInitInstance;
+    return globalLoggingInitInstance;
+}
+
+// To ensure that the combined struct defined above is initialized even
+// if the initialization guard function wasn't called, a dummy static
+// instance of a struct is defined below, which will call the
+// initialization guard function.
+//
+struct GlobalLoggingInitCall
+{
+    GlobalLoggingInitCall()
+    {
+        getGlobalLoggingInitStruct();
+    }
+};
+
+static GlobalLoggingInitCall globalLoggingInitCall;
+
+static LogTagManager& getLogTagManager()
+{
+    static LogTagManager& logTagManagerInstance = getGlobalLoggingInitStruct().logTagManager;
+    return logTagManagerInstance;
 }
 
 static LogLevel& getLogLevelVariable()
 {
-    static LogLevel g_logLevel = parseLogLevelConfiguration();
-    return g_logLevel;
+    static LogLevel& refGlobalLogLevel = getGlobalLogTag()->level;
+    return refGlobalLogLevel;
+}
+
+LogTag* getGlobalLogTag()
+{
+    static LogTag* globalLogTagPtr = getGlobalLoggingInitStruct().logTagManager.get("global");
+    return globalLogTagPtr;
+}
+
+} // namespace
+
+void registerLogTag(LogTag* plogtag)
+{
+    if (!plogtag || !plogtag->name)
+    {
+        return;
+    }
+    internal::getLogTagManager().assign(plogtag->name, plogtag);
+}
+
+void setLogTagLevel(const char* tag, LogLevel level)
+{
+    if (!tag)
+    {
+        return;
+    }
+    internal::getLogTagManager().setLevelByFullName(std::string(tag), level);
+}
+
+LogLevel getLogTagLevel(const char* tag)
+{
+    if (!tag)
+    {
+        return getLogLevel();
+    }
+    const LogTag* ptr = internal::getLogTagManager().get(std::string(tag));
+    if (!ptr)
+    {
+        return getLogLevel();
+    }
+    return ptr->level;
 }
 
 LogLevel setLogLevel(LogLevel logLevel)
 {
-    LogLevel old = getLogLevelVariable();
-    getLogLevelVariable() = logLevel;
+    // note: not thread safe, use sparingly and do not critically depend on outcome
+    LogLevel& refGlobalLevel = internal::getLogLevelVariable();
+    const LogLevel old = refGlobalLevel;
+    refGlobalLevel = logLevel;
     return old;
 }
 
 LogLevel getLogLevel()
 {
-    return getLogLevelVariable();
+    return internal::getLogLevelVariable();
 }
 
 namespace internal {
@@ -105,6 +211,29 @@ void writeLogMessage(LogLevel logLevel, const char* message)
         (*out) << std::flush;
 }
 
+void writeLogMessageEx(LogLevel logLevel, const char* tag, const char* file, int line, const char* func, const char* message)
+{
+    std::ostringstream strm;
+    if (tag)
+    {
+        strm << tag << " ";
+    }
+    if (file)
+    {
+        strm << file << " ";
+    }
+    if (line > 0)
+    {
+        strm << "(" << line << ") ";
+    }
+    if (func)
+    {
+        strm << func << " ";
+    }
+    strm << message;
+    writeLogMessage(logLevel, strm.str().c_str());
+}
+
 } // namespace
 
 }}} // namespace

modules/core/src/utils/logtagconfig.hpp

+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef OPENCV_CORE_LOGTAGCONFIG_HPP
+#define OPENCV_CORE_LOGTAGCONFIG_HPP
+
+#if 1 // if not already in precompiled headers
+#include <opencv2/core/utils/logger.defines.hpp>
+#include <string>
+#endif
+
+namespace cv {
+namespace utils {
+namespace logging {
+
+struct LogTagConfig
+{
+    std::string namePart;
+    LogLevel level;
+    bool isGlobal;
+    bool hasPrefixWildcard;
+    bool hasSuffixWildcard;
+
+    LogTagConfig()
+        : namePart()
+        , level()
+        , isGlobal()
+        , hasPrefixWildcard()
+        , hasSuffixWildcard()
+    {
+    }
+
+    LogTagConfig(const std::string& _namePart, LogLevel _level, bool _isGlobal = false,
+        bool _hasPrefixWildcard = false, bool _hasSuffixWildcard = false)
+        : namePart(_namePart)
+        , level(_level)
+        , isGlobal(_isGlobal)
+        , hasPrefixWildcard(_hasPrefixWildcard)
+        , hasSuffixWildcard(_hasSuffixWildcard)
+    {
+    }
+
+    LogTagConfig(const LogTagConfig&) = default;
+    LogTagConfig(LogTagConfig&&) = default;
+    ~LogTagConfig() = default;
+};
+
+}}}
+
+#endif

modules/core/src/utils/logtagconfigparser.cpp

+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#include "../precomp.hpp"
+#include "logtagconfigparser.hpp"
+
+namespace cv {
+namespace utils {
+namespace logging {
+
+LogTagConfigParser::LogTagConfigParser()
+{
+    m_parsedGlobal.namePart = "global";
+    m_parsedGlobal.isGlobal = true;
+    m_parsedGlobal.hasPrefixWildcard = false;
+    m_parsedGlobal.hasSuffixWildcard = false;
+    m_parsedGlobal.level = LOG_LEVEL_VERBOSE;
+}
+
+LogTagConfigParser::LogTagConfigParser(const std::string& input)
+{
+    parse(input);
+}
+
+LogTagConfigParser::~LogTagConfigParser()
+{
+}
+
+bool LogTagConfigParser::parse(const std::string& input)
+{
+    m_input = input;
+    segmentTokens();
+    return (m_malformed.empty());
+}
+
+bool LogTagConfigParser::hasMalformed() const
+{
+    return !m_malformed.empty();
+}
+
+const LogTagConfig& LogTagConfigParser::getGlobalConfig() const
+{
+    return m_parsedGlobal;
+}
+
+const std::vector<LogTagConfig>& LogTagConfigParser::getFullNameConfigs() const
+{
+    return m_parsedFullName;
+}
+
+const std::vector<LogTagConfig>& LogTagConfigParser::getFirstPartConfigs() const
+{
+    return m_parsedFirstPart;
+}
+
+const std::vector<LogTagConfig>& LogTagConfigParser::getAnyPartConfigs() const
+{
+    return m_parsedAnyPart;
+}
+
+const std::vector<std::string>& LogTagConfigParser::getMalformed() const
+{
+    return m_malformed;
+}
+
+void LogTagConfigParser::segmentTokens()
+{
+    const size_t len = m_input.length();
+    std::vector<std::pair<size_t, size_t>> startStops;
+    bool wasSeparator = true;
+    for (size_t pos = 0u; pos < len; ++pos)
+    {
+        char c = m_input[pos];
+        bool isSeparator = (c == ' ' || c == '\t' || c == ';');
+        if (!isSeparator)
+        {
+            if (wasSeparator)
+            {
+                startStops.emplace_back(pos, pos + 1u);
+            }
+            else
+            {
+                startStops.back().second = pos + 1u;
+            }
+        }
+        wasSeparator = isSeparator;
+    }
+    for (const auto& startStop : startStops)
+    {
+        const auto s = m_input.substr(startStop.first, startStop.second - startStop.first);
+        parseNameAndLevel(s);
+    }
+}
+
+void LogTagConfigParser::parseNameAndLevel(const std::string& s)
+{
+    const size_t npos = std::string::npos;
+    const size_t len = s.length();
+    size_t colonIdx = s.find_first_of(":=");
+    if (colonIdx == npos)
+    {
+        // See if the whole string is a log level
+        auto parsedLevel = parseLogLevel(s);
+        if (parsedLevel.second)
+        {
+            // If it is, assume the log level is for global
+            parseWildcard("", parsedLevel.first);
+            return;
+        }
+        else
+        {
+            // not sure what to do.
+            m_malformed.push_back(s);
+            return;
+        }
+    }
+    if (colonIdx == 0u || colonIdx + 1u == len)
+    {
+        // malformed (colon or equal sign at beginning or end), cannot do anything
+        m_malformed.push_back(s);
+        return;
+    }
+    size_t colonIdx2 = s.find_first_of(":=", colonIdx + 1u);
+    if (colonIdx2 != npos)
+    {
+        // malformed (more than one colon or equal sign), cannot do anything
+        m_malformed.push_back(s);
+        return;
+    }
+    auto parsedLevel = parseLogLevel(s.substr(colonIdx + 1u));
+    if (parsedLevel.second)
+    {
+        parseWildcard(s.substr(0u, colonIdx), parsedLevel.first);
+        return;
+    }
+    else
+    {
+        // Cannot recognize the right side of the colon or equal sign.
+        // Not sure what to do.
+        m_malformed.push_back(s);
+        return;
+    }
+}
+
+void LogTagConfigParser::parseWildcard(const std::string& name, LogLevel level)
+{
+    constexpr size_t npos = std::string::npos;
+    const size_t len = name.length();
+    if (len == 0u)
+    {
+        m_parsedGlobal.level = level;
+        return;
+    }
+    const bool hasPrefixWildcard = (name[0u] == '*');
+    if (hasPrefixWildcard && len == 1u)
+    {
+        m_parsedGlobal.level = level;
+        return;
+    }
+    const size_t firstNonWildcard = name.find_first_not_of("*.");
+    if (hasPrefixWildcard && firstNonWildcard == npos)
+    {
+        m_parsedGlobal.level = level;
+        return;
+    }
+    const bool hasSuffixWildcard = (name[len - 1u] == '*');
+    const size_t lastNonWildcard = name.find_last_not_of("*.");
+    std::string trimmedNamePart = name.substr(firstNonWildcard, lastNonWildcard - firstNonWildcard + 1u);
+    // The case of a single asterisk has been handled above;
+    // here we only handle the explicit use of "global" in the log config string.
+    const bool isGlobal = (trimmedNamePart == "global");
+    if (isGlobal)
+    {
+        m_parsedGlobal.level = level;
+        return;
+    }
+    LogTagConfig result(trimmedNamePart, level, false, hasPrefixWildcard, hasSuffixWildcard);
+    if (hasPrefixWildcard)
+    {
+        m_parsedAnyPart.emplace_back(std::move(result));
+    }
+    else if (hasSuffixWildcard)
+    {
+        m_parsedFirstPart.emplace_back(std::move(result));
+    }
+    else
+    {
+        m_parsedFullName.emplace_back(std::move(result));
+    }
+}
+
+std::pair<LogLevel, bool> LogTagConfigParser::parseLogLevel(const std::string& s)
+{
+    const auto falseDontCare = std::make_pair(LOG_LEVEL_VERBOSE, false);
+    const auto make_parsed_result = [](LogLevel lev) -> std::pair<LogLevel, bool>
+    {
+        return std::make_pair(lev, true);
+    };
+    const size_t len = s.length();
+    if (len >= 1u)
+    {
+        const char c = (char)std::toupper(s[0]);
+        switch (c)
+        {
+        case '0':
+            if (len == 1u)
+            {
+                return make_parsed_result(LOG_LEVEL_SILENT);
+            }
+            break;
+        case 'D':
+            if (len == 1u ||
+                (len == 5u && cv::toUpperCase(s) == "DEBUG"))
+            {
+                return make_parsed_result(LOG_LEVEL_DEBUG);
+            }
+            if ((len == 7u && cv::toUpperCase(s) == "DISABLE") ||
+                (len == 8u && cv::toUpperCase(s) == "DISABLED"))
+            {
+                return make_parsed_result(LOG_LEVEL_SILENT);
+            }
+            break;
+        case 'E':
+            if (len == 1u ||
+                (len == 5u && cv::toUpperCase(s) == "ERROR"))
+            {
+                return make_parsed_result(LOG_LEVEL_ERROR);
+            }
+            break;
+        case 'F':
+            if (len == 1u ||
+                (len == 5u && cv::toUpperCase(s) == "FATAL"))
+            {
+                return make_parsed_result(LOG_LEVEL_FATAL);
+            }
+            break;
+        case 'I':
+            if (len == 1u ||
+                (len == 4u && cv::toUpperCase(s) == "INFO"))
+            {
+                return make_parsed_result(LOG_LEVEL_INFO);
+            }
+            break;
+        case 'O':
+            if (len == 3u && cv::toUpperCase(s) == "OFF")
+            {
+                return make_parsed_result(LOG_LEVEL_SILENT);
+            }
+            break;
+        case 'S':
+            if (len == 1u ||
+                (len == 6u && cv::toUpperCase(s) == "SILENT"))
+            {
+                return make_parsed_result(LOG_LEVEL_SILENT);
+            }
+            break;
+        case 'V':
+            if (len == 1u ||
+                (len == 7u && cv::toUpperCase(s) == "VERBOSE"))
+            {
+                return make_parsed_result(LOG_LEVEL_VERBOSE);
+            }
+            break;
+        case 'W':
+            if (len == 1u ||
+                (len == 4u && cv::toUpperCase(s) == "WARN") ||
+                (len == 7u && cv::toUpperCase(s) == "WARNING") ||
+                (len == 8u && cv::toUpperCase(s) == "WARNINGS"))
+            {
+                return make_parsed_result(LOG_LEVEL_WARNING);
+            }
+            break;
+        default:
+            break;
+        }
+        // fall through
+    }
+    return falseDontCare;
+}
+
+std::string LogTagConfigParser::toString(LogLevel level)
+{
+    switch (level)
+    {
+    case LOG_LEVEL_SILENT:
+        return "SILENT";
+    case LOG_LEVEL_FATAL:
+        return "FATAL";
+    case LOG_LEVEL_ERROR:
+        return "ERROR";
+    case LOG_LEVEL_WARNING:
+        return "WARNING";
+    case LOG_LEVEL_INFO:
+        return "INFO";
+    case LOG_LEVEL_DEBUG:
+        return "DEBUG";
+    case LOG_LEVEL_VERBOSE:
+        return "VERBOSE";
+    default:
+        return std::to_string((int)level);
+    }
+}
+
+}}} //namespace

modules/core/src/utils/logtagconfigparser.hpp

+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef OPENCV_CORE_LOGTAGCONFIGPARSER_HPP
+#define OPENCV_CORE_LOGTAGCONFIGPARSER_HPP
+
+#if 1 // if not already in precompiled headers
+#include <string>
+#include <vector>
+#include <functional>
+#endif
+
+#include <opencv2/core/utils/logtag.hpp>
+#include "logtagconfig.hpp"
+
+namespace cv {
+namespace utils {
+namespace logging {
+
+class LogTagConfigParser
+{
+public:
+    LogTagConfigParser();
+    explicit LogTagConfigParser(const std::string& input);
+    ~LogTagConfigParser();
+
+public:
+    bool parse(const std::string& input);
+    bool hasMalformed() const;
+    const LogTagConfig& getGlobalConfig() const;
+    const std::vector<LogTagConfig>& getFullNameConfigs() const;
+    const std::vector<LogTagConfig>& getFirstPartConfigs() const;
+    const std::vector<LogTagConfig>& getAnyPartConfigs() const;
+    const std::vector<std::string>& getMalformed() const;
+
+private:
+    void segmentTokens();
+    void parseNameAndLevel(const std::string& s);
+    void parseWildcard(const std::string& name, LogLevel level);
+    static std::pair<LogLevel, bool> parseLogLevel(const std::string& s);
+    static std::string toString(LogLevel level);
+
+private:
+    std::string m_input;
+    LogTagConfig m_parsedGlobal;
+    std::vector<LogTagConfig> m_parsedFullName;
+    std::vector<LogTagConfig> m_parsedFirstPart;
+    std::vector<LogTagConfig> m_parsedAnyPart;
+    std::vector<std::string> m_malformed;
+};
+
+}}} //namespace
+
+#endif

modules/core/src/utils/logtagmanager.hpp

+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html.
+
+#ifndef OPENCV_CORE_LOGTAGMANAGER_HPP
+#define OPENCV_CORE_LOGTAGMANAGER_HPP
+
+#if 1 // if not already in precompiled headers
+#include <string>
+#include <algorithm>
+#include <functional>
+#include <unordered_map>
+#include <mutex>
+#include <memory>
+#endif
+
+#include <opencv2/core/utils/logtag.hpp>
+#include "logtagconfig.hpp"
+
+namespace cv {
+namespace utils {
+namespace logging {
+
+// forward declaration
+class LogTagConfigParser;
+
+// A lookup table of LogTags using full name, first part of name, and any part of name.
+// The name parts of a LogTag is delimited by period.
+//
+// This class does not handle wildcard characters. The name-matching method can only be
+// selected by calling the appropriate function.
+//
+class LogTagManager
+{
+private:
+    // Current implementation does not seem to require recursive mutex;
+    // also, extensible functions (accepting user-provided callback) are not allowed
+    // to call LogTagManger (to prevent iterator invalidation), which needs enforced
+    // with a non-recursive mutex.
+    using MutexType = std::mutex;
+    using LockType = std::lock_guard<MutexType>;
+
+    enum class MatchingScope
+    {
+        None,
+        Full,
+        FirstNamePart,
+        AnyNamePart
+    };
+
+    struct ParsedLevel
+    {
+        LogLevel level;
+        MatchingScope scope;
+
+        ParsedLevel()
+            : level()
+            , scope(MatchingScope::None)
+        {
+        }
+    };
+
+    struct FullNameInfo
+    {
+        LogTag* logTagPtr;
+        ParsedLevel parsedLevel;
+    };
+
+    struct NamePartInfo
+    {
+        ParsedLevel parsedLevel;
+    };
+
+    struct CrossReference
+    {
+        size_t m_fullNameId;
+        size_t m_namePartId;
+        size_t m_matchingPos;
+        FullNameInfo* m_fullNameInfo;
+        NamePartInfo* m_namePartInfo;
+
+        explicit CrossReference(size_t fullNameId, size_t namePartId, size_t matchingPos,
+            FullNameInfo* fullNameInfo, NamePartInfo* namePartInfo)
+            : m_fullNameId(fullNameId)
+            , m_namePartId(namePartId)
+            , m_matchingPos(matchingPos)
+            , m_fullNameInfo(fullNameInfo)
+            , m_namePartInfo(namePartInfo)
+        {
+        }
+    };
+
+    struct FullNameLookupResult
+    {
+        // The full name being looked up
+        std::string m_fullName;
+
+        // The full name being broken down into name parts
+        std::vector<std::string> m_nameParts;
+
+        // The full name ID that is added or looked up from the table
+        size_t m_fullNameId;
+
+        // The name part IDs that are added to or looked up from the table
+        // listed in the same order as m_nameParts
+        std::vector<size_t> m_namePartIds;
+
+        // The information struct for the full name
+        FullNameInfo* m_fullNameInfoPtr;
+
+        // Specifies whether cross references (full names that match the name part)
+        // should be computed.
+        bool m_findCrossReferences;
+
+        // List of all full names that match the given name part.
+        // This field is computed only if m_findCrossReferences is true.
+        std::vector<CrossReference> m_crossReferences;
+
+        explicit FullNameLookupResult(const std::string& fullName)
+            : m_fullName(fullName)
+            , m_nameParts()
+            , m_fullNameId()
+            , m_namePartIds()
+            , m_fullNameInfoPtr()
+            , m_findCrossReferences()
+            , m_crossReferences()
+        {
+        }
+    };
+
+    struct NamePartLookupResult
+    {
+        // The name part being looked up
+        std::string m_namePart;
+
+        // The name part ID that is added or looked up from the table
+        size_t m_namePartId;
+
+        // Information struct ptr for the name part
+        NamePartInfo* m_namePartInfoPtr;
+
+        // Specifies whether cross references (full names that match the name part) should be computed.
+        bool m_findCrossReferences;
+
+        // List of all full names that match the given name part.
+        // This field is computed only if m_findCrossReferences is true.
+        std::vector<CrossReference> m_crossReferences;
+
+        explicit NamePartLookupResult(const std::string& namePart)
+            : m_namePart(namePart)
+            , m_namePartId()
+            , m_namePartInfoPtr()
+            , m_findCrossReferences()
+            , m_crossReferences()
+        {
+        }
+    };
+
+    struct NameTable
+    {
+        // All data structures in this class are append-only. The item count
+        // is being used as an incrementing integer key.
+
+    public:
+        // Full name information struct.
+        std::vector<FullNameInfo> m_fullNameInfos;
+
+        // Name part information struct.
+        std::vector<NamePartInfo> m_namePartInfos;
+
+        // key: full name (string)
+        // value: full name ID
+        // .... (index into the vector of m_fullNameInfos)
+        // .... (key into m_fullNameToNamePartIds)
+        // .... (value.second in m_namePartToFullNameIds)
+        std::unordered_map<std::string, size_t> m_fullNameIds;
+
+        // key: name part (string)
+        // value: name part ID
+        // .... (index into the vector of m_namePartInfos)
+        // .... (key into m_namePartToFullNameIds)
+        // .... (value.second in m_fullNameToNamePartIds)
+        std::unordered_map<std::string, size_t> m_namePartIds;
+
+        // key: full name ID
+        // value.first: name part ID
+        // value.second: occurrence position of name part in the full name
+        std::unordered_multimap<size_t, std::pair<size_t, size_t>> m_fullNameToNamePartIds;
+
+        // key: name part ID
+        // value.first: full name ID
+        // value.second: occurrence position of name part in the full name
+        std::unordered_multimap<size_t, std::pair<size_t, size_t>> m_namePartToFullNameIds;
+
+    public:
+        void addOrLookupFullName(FullNameLookupResult& result);
+        void addOrLookupNamePart(NamePartLookupResult& result);
+        FullNameInfo* getFullNameInfo(const std::string& fullName);
+
+    private:
+        // Add or get full name. Does not compute name parts or access them in the table.
+        // Returns the full name ID, and a bool indicating if the full name is new.
+        std::pair<size_t, bool> internal_addOrLookupFullName(const std::string& fullName);
+
+        // Add or get multiple name parts. Saves name part IDs into a vector.
+        void internal_addOrLookupNameParts(const std::vector<std::string>& nameParts, std::vector<size_t>& namePartIds);
+
+        // Add or get name part. Returns namePartId.
+        size_t internal_addOrLookupNamePart(const std::string& namePart);
+
+        // For each name part ID, insert the tuples (full name, name part ID, name part position)
+        // into the cross reference table
+        void internal_addCrossReference(size_t fullNameId, const std::vector<size_t>& namePartIds);
+
+        // Gather pointer for full name info struct.
+        // Note: The pointer is interior to the table vector. The pointers are invalidated
+        // if the table is modified.
+        FullNameInfo* internal_getFullNameInfo(size_t fullNameId);
+
+        // Gather pointers for name part info struct.
+        // Note: The pointers are interior to the table vector. The pointers are invalidated
+        // if the table is modified.
+        NamePartInfo* internal_getNamePartInfo(size_t namePartId);
+
+        void internal_findMatchingNamePartsForFullName(FullNameLookupResult& fullNameResult);
+        void internal_findMatchingFullNamesForNamePart(NamePartLookupResult& result);
+    };
+
+public:
+    LogTagManager(LogLevel defaultUnconfiguredGlobalLevel);
+    ~LogTagManager();
+
+public:
+    // Parse and apply the config string.
+    void setConfigString(const std::string& configString, bool apply = true);
+
+    // Gets the config parser. This is necessary to retrieve the list of malformed strings.
+    LogTagConfigParser& getConfigParser() const;
+
+    // Add (register) the log tag.
+    // Note, passing in nullptr as value is equivalent to unassigning.
+    void assign(const std::string& fullName, LogTag* ptr);
+
+    // Unassign the log tag. This is equivalent to calling assign with nullptr value.
+    void unassign(const std::string& fullName);
+
+    // Retrieve the log tag by exact name.
+    LogTag* get(const std::string& fullName);
+
+    // Changes the log level of the tag having the exact full name.
+    void setLevelByFullName(const std::string& fullName, LogLevel level);
+
+    // Changes the log level of the tags matching the first part of the name.
+    void setLevelByFirstPart(const std::string& firstPart, LogLevel level);
+
+    // Changes the log level of the tags matching any part of the name.
+    void setLevelByAnyPart(const std::string& anyPart, LogLevel level);
+
+    // Changes the log level of the tags with matching name part according
+    // to the specified scope.
+    void setLevelByNamePart(const std::string& namePart, LogLevel level, MatchingScope scope);
+
+private:
+    bool internal_applyFullNameConfigToTag(FullNameInfo& fullNameInfo);
+    bool internal_applyNamePartConfigToSpecificTag(FullNameLookupResult& fullNameResult);
+    void internal_applyNamePartConfigToMatchingTags(NamePartLookupResult& namePartResult);
+
+private:
+    static std::vector<std::string> splitNameParts(const std::string& fullName);
+    static bool internal_isNamePartMatch(MatchingScope scope, size_t matchingPos);
+
+private:
+    static constexpr const char* m_globalName = "global";
+
+private:
+    mutable MutexType m_mutex;
+    std::unique_ptr<LogTag> m_globalLogTag;
+    NameTable m_nameTable;
+    std::shared_ptr<LogTagConfigParser> m_config;
+};
+
+}}} //namespace
+
+#endif //OPENCV_CORE_LOGTAGMANAGER_HPP