Log Monitoring
- Log monitoring is to match keywords, traffic, response time occurring in real time in an acquisition cycle according to full log path configured by the user. It mainly solves the problem of low service performance and can preliminarily position the boundary of the problems.
- Log acquisition will not read old log data. Only after configured keywords appear, can corresponding monitoring indicators generate. If monitoring indicators cannot be found all the way, please check whether corresponding keywords appear in the log in corresponding period.
- Please ensure machines under selected system or application all have the log, otherwise monitoring will go wrong.
Operation Guide
Step 1: Select [Intelligent Monitoring] - [Monitoring Configuration] in the menu, select nodes at or below product lines in the service tree at the left side, click New on the Acquisition Configuration page as shown in the following figure.
Step 2: Open acquisition configuration new window as shown in the figure. In which,
Monitoring Type: Select log monitoring
Name: Name of this acquisition task, which will be regarded as the prefix of monitoring indicator. It supports letters, numbers and _, and it must begin with a letter
Scope: Scope of log acquisition is associated with selected nodes in the service tree, with the principle of being selectable downward. When the application node in the service tree is selected, only "Application" can be selected in the scope; when system node in the service tree is selected, "System" or "Application" can be selected in the scope; when product line node in the service tree is selected, "Product Line", "System", "Application" can be selected in the scope
Node: Specific node of process collection configuration, when selecting a system or product line node, all the applications that the node belongs to will be replaced.
Acquisition cycle: Select acquisition cycle to be configured.
Log path: Enter log path
Monitoring Contents:
Log type: Form supports selection error or traffic
Contain/not contain: Add keywords matching contain or not contain
Name monitoring items: Name the result generated by this log monitoring, i.e., customize the name of a monitoring item that is easy to be recognized (cannot start with numbers)
Description: Add description for this log monitoring
Value in the case of no data: Set a default value which is the value of monitoring item when no keywords have been matched.
When alarm is triggered, return to log field, which refers to log row that triggers the alarm displayed in the alarm notification.
JSON Configuration Case
(1) Quick start of log acquisition
When only monitoring simple keywords in a log, the following is an example of an acquisition configuration:
json
{
"method": "log",
"cycle": 60,
"para": {
"logPath": "/A/B/C/d.ERROR",
"filters": [
{
"matchStr": "can not follow scaler:",
"items": [
{
"classify":"errno",
"metricName": "followscaler_error",
"value": "",
"tags": "",
"desc":"Description of the log monitoring indicators",
"scene":false,
"defaultValue": 0
}
]
},
{
"matchStr": "getsockopt: connection refused",
"items": [
{
"classify":"errno",
"metricName": "ping_error",
"value": "",
"tags": "",
"desc":"Description of the log monitoring indicators",
"scene":false,
"defaultValue": 0
}
]
}
]
}
}
Compulsory part:
logPath: Fill in full path of logs.
matchStr: Fill in keywords of logs.
metricName: Please name the monitoring item produced by such keyword and fill in the name. Only English letters, numbers, underline (_) are supported for the name, and the name cannot begin with a number.
tags: Please fill in the keys in the produced monitoring indicators, and separate them with the mark ",". The key can be named with English letters, numbers, "、_" (underlines), "." (period in English) and "-" (line-through in English).
value: The completed contents refer to the values to be monitored. The null content by default refers to the computing PV. However, cost and other numeric values can be filled in.
Explanation for other fields: Cycle refers to acquisition cycle, count the times of keyword appearance in each acquisition cycle. defaultValue refers to providing a default value that usually is 0 when no configured keyword can be found in a cycle.
Other important supplements: If multiple keywords in the same log need to be monitored, we support the array form of keyword configuration, adding this field more than once in the filter (note: separate by comma)
(2) Multipath Configuration Method
When there are multiple logs in the same system or application and the log types and the keywords to be monitored are the same, multiple log paths can be added in one configuration according to the following examples.
json
{
"method": "log",
"para": {
"multiLogFilePath": {
"logFilePattern": "/a/b/c/$ulog_pattern$/d.log",
"patternValues": [
{
"value": "b2b.abcd.com",
"trans": "b2b.abcd"
},
{
"value": "bbj.abcd.com",
"trans": "bbj.abcd"
},
{
"value": "book.abcd.com",
"trans": "book.abcd"
},
{
"value": "buyer.b2b.abcd.com",
"trans": "buyer.b2b.abcd"
}
]
},
"filters": [
{
"items": [
{
"classify":"latency",
"metricName": "res_time",
"value": "res_time",
"tags": "res_code"
}
],
"matchStr": ".*\\|\\|(?P<res_code>\\d+)\\|\\|(?P<res_tim>[\\.\\d]+)\\|\\|\\S+$"
}
]
},
"cycle": 60
}
Fill in logFilePattern with common parts of each log path, replace the different parts with $ulog_pattern$. Fill in patternValues with different parts of each log path, value is original content, and trans is used to be converted into other descriptions when the original content is too long.
(3) Use of prematch
The function of prematch is to filter out the log contents that may have keywords according to the fields in prematch firstly when extracting the keywords, and then match the Regular parts. Regular is not supported in prematch. In addition, there are also preNotMatch fields, which mean that not matching the keywords filled in, and Regular is also not supported.
The following is an example using the prematch configuration:
json
{
"method": "log",
"para": {
"logPath": "/a/b/c/error.log",
"filters": [
{
"preMatch": "Connect to mysql fail",
"items": [
{
"classify":"errno",
"metricName": "Connectfail",
"defaultValue": 0,
"value": "",
"tags": ""
}
],
"matchStr": "^."
},
{
"preMatch": "task not exists in db id",
"items": [
{
"classify":"errno",
"metricName": "task_not_exists",
"defaultValue": 0,
"value": "",
"tags": ""
}
],
"matchStr": "^."
}
]
},
"cycle": 60
}
(4) Configuration method of dynamic unknown path
The wildcards that support dynamic paths are $DEPLOY_PATH$, $INS_NAME$, $APP$, etc. The latter two are used when the log path changes with the APP name and instance name.
There are two scenarios for using $DEPLOY_PATH$:
-
When the log path is a dynamically changing path, the fixed path cannot be filled in. In this situation, it is required to use $DEPLOY_PATH$ field to match the path.
-
When there are many applications (it is tedious to match applications with the path one by one) and the logs of each application need to be monitored, $DEPLOY_PATH$ can be used to configure all log collections at higher nodes. The premise is that it is required to configure the log paths of each application node in the service tree.
The following is the configuration template for a dynamic unknown path:
json
{
"method": "log",
"para": {
"logPath": "$DEPLOY_PATH$",
"filters": [
{
"items": [
{
"classify":"errno",
"metricName": "jss_access_error",
"defaultValue": 0,
"value": "",
"tags": ""
}
],
"matchStr": "Failed"
}
]
},
"cycle": 60
}
(5) Configuration methods for filtering APP or group
When the collection configuration is for the part of APPs under a system or the part of groups under an APP, "apps" or "groups" field can be used to filter. The specific template is as follows:
json
{
"method": "log",
"cycle": 60,
"apps":"Fill out the APPName. If there are multiple APPNames, separate them by English comma.",
"para": {
"logPath": "/a/b/c.ERROR",
"filters": [
{
"matchStr": "can not follow scaler:",
"items": [
{
"classify":"errno",
"metricName": "followscaler_error",
"value": "",
"tags": "",
"defaultValue": 0
}
]
},
{
"matchStr": "getsockopt: connection refused",
"items": [
{
"classify":"errno",
"metricName": "ping_error",
"value": "",
"tags": "",
"defaultValue": 0
}
]
}
]
}
}
开始与售前顾问沟通
可直接拨打电话 400-098-8505转1
我们的产品专家为您找到最合适的产品/解决⽅案
1v1线上咨询获取售前专业咨询
专业产品顾问，随时随地沟通
