Searching logs with Kibana
The KibanaLogs screen allows you to search the Elasticsearch database for information using the Kibana search query syntax, which is a modified version of the Lucene query syntax.
Typical use cases
Kibana is ideal for searching through large amounts of granular log data with open-ended queries. For example, displaying all instances of an IP address across the entire database of log records for a specified time period. Kibana will return the full details of all logs containing the specified IP address.
Kibana query builder and syntax
The Kibana Logs screen uses the Kibana syntax (the current supported version of Kibana is 7.10). Kibana is an open source tool developed in 2013 by the company Elastic. A Kibana tutorial is available from the Elastic company website. The main search bar features a "query builder" with a dropdown showing valid query syntax, fields and terms. The following table provides a guide to using the proper Kibana syntax when searching in :
| Valid Query Format | |
| Exact Match entire field | field1:"term" |
| Field contains A or B | field1:(term1 OR term2) |
| Field contains A and B | field1:(term 1 AND term2) |
| Field missing | -field:* |
| Field exists_ | exists_:title |
| Ranges | |
| 1 to 5, including 1 and 5 | [1 TO 5] |
| 1 to 5, excluding 1 and 5 | {1 TO 5} |
| 1 to 5, including 1, excluding 5 | [1 TO 5} |
| Larger than | > |
| Smaller than | < |
| Larger or equal to | >= |
| Smaller or equal to | <= |
| IP CIDR notation | "10.0.0.0/8" |
| Boolean Operations | |
| Must be present | +
|
| Must not be present | - |
| And | AND&& |
| or | OR|| |
| Not | NOT| |
| Grouping | |
| Basics (term operator term) operator term Boolean logic combined with brackets (term1 OR term2) AND term3 Must contain either or both term1/term2 and term3 | |
| Regular expressions | |
| Regular expression patterns can be embedded in the query string by wrapping them in forward-slash ("/") | Field:/joh?n(ath[oa]n)/ |
| Wildcards | |
| Single character | ? |
| Multiple characters | * |
| Fuzzy | ~ |
primary search entities
src_host – The device to which the log pertains to (the opposite being dst_host).user_name – The log pertaining to the user.
src_ip4 – The IP address pertaining to the device (the opposite being dst_ip4).
src_mac – The MAC address of the device or device's network adaptor to which the log pertains.
user_mail/sender/recipient – The email address of the user to which the log pertains.
dvc_host – The device that logged the event.
Example queries
data_source:windows_security_events AND event_id:4625 AND (user_name:administrator OR user_name:admin)
data_source:cisco_asa_firewall AND direction:inbound AND (dst_port:3389 OR dst_port:445 OR dst_port:22)
data_source:tipping_point_ips AND rule_name:"19698: bittorrent: udptracker connect request" AND src_ip4:"151.193.0.0/16"
data_source:windows_security_events AND event_id:4688 AND process_path:"c:\\windows\\system32\\cmd.exe"
Kibana log search management functions
The Search bar at the top of the screen specifies the search query for a selected time frame and uses lucene query syntax. The Refresh button re-runs the search query currently specified in the Search bar. Click the calendar dropdown to specify a timeframe for the search, either past or future, or choose from one of the preset options, and program an automatic refresh period for the search.
Clicking New creates a new search based on the parameters that have been entered.
Clicking Save saves the current search so it can be re-used.
Clicking Open displays a list of previously saved searches to select and run.
Clicking Refresh Field refreshes all fields used in a search.
Clicking Export to CSV allows you to export and download the search results to a file in CSV format to a maximum of 1,048,574 records (or rows), which is the absolute limit supported by Microsoft's Excel application.
Clicking Auto-refresh allows you to enable an automatic refresh of fields based on a preset time interval.
Clicking the Search Time Span button to the far right of the interface presents options for setting the time period when displaying log search results.
-
Quick
-
Relative
-
Absolute
Filter options
On the left side of the Logs screen is an interface to select field columns to display in the search results. The Search field here is for finding and filtering the many different log fields that contain information used by to trigger Indicators and Detections. Available fields can be selected to include information about them in a column of the search results.
Clicking the Filter by type button displays an interface to filter fields by Aggregatable, Searchable, and Type. There is also a toggle to Hide missing Fields from the list of selectable fields.
Clicking the +Add filter button displays options for specifying a precise field filter for the search.
Click the Edit as Query DSL link to open an interface to use Elasticsearch's full Query DSL (Domain Specific Language) based on JSON to define queries, or use the Field and Operator dropdown to define the field filter.