Adding NSS Feeds for Web Logs


Adding NSS Feeds for Web Logs

You can configure up to eight NSS feeds that define the web logs that the NSS sends to the SIEM. You can also configure multiple types of filters. For example, if an admin selects the location HQ and department Finance, the NSS will select logs that belong to both the HQ location and Finance department. Note that a large number of filters or complex filters, such as string searches, might impact the performance of the NSS.

Before you start configuring a feed for web logs, consider the guidelines for configuring feeds.

To configure a feed for web logs:

  1. Go to Administration > Settings > Nanolog Streaming Service.
  2. From the NSS Feeds tab, click Add NSS Feed and complete the following:
    • Feed Name: Enter or edit the name of the feed. Each feed is a connection between the NSS and your SIEM.
    • NSS TypeNSS for Web is selected by default.
    • NSS Server: Choose an NSS from the list.
    • Status: The NSS feed is Enabled by default. Choose Disabled if you want to activate it at a later time.
    • SIEM IP Address and SIEM TCP Port: Enter the IP address and port of the SIEM to which the logs are streamed. Ensure that the SIEM is configured to accept the feed from the NSS.
    • Log Type: Choose Web Log.
    • Feed Output Type: The output is a comma-separated (CSV) list by default. Choose Tab-separated to create a tab-separated list. Choose Custom to use a different delimiter, such as a dash, and enter the delimiter when you specify the feed output format. This menu also lists feed output types for specific SIEMs.
    • Feed Escape Character: The Zscaler service hex encodes all non-printable ASCII characters that are in URLs when it sends the logs to the NSS. Any URL character that is less than 0x21, or above 0x7E, will be encoded as %HH. This ensures that your SIEM will be able to parse the URLs in case they contain non-printable characters. For example, a \n char in a URL is encoded as %0A, and a space is encoded as %20. In this field, you can specify additional characters that you would like to encode. For example, type a comma (,) to encode it as %2C. This is useful if you are using this character as your delimiter and would like to ensure it does not cause erroneous delimitation. Note that the service encodes characters in URLs, host names, and referer URLs only. If custom encoding was done for a record, the %s{eedone} field will be YES for that record.
    • Feed Output Format: These are the fields that will be displayed in the output. You can edit the default list and if you chose Custom as the Field Output Type, change the delimiter as well. See NSS Feed Output Format for information about the available fields and their syntax.
    • User Obfuscation: You can enable user obfuscation. When you do, it displays a random string instead of the user names. If this is enabled, the ‘login’ field in Feed Format Output automatically changes to ‘ologin’ field which outputs the obfuscated login name. Choose Disable to display the user names.
    • Timezone: By default, this is set to the organization's time zone. The time zone you set applies to the time field in the output file. The time zone automatically adjusts to changes in daylight savings in the specific time zone. The configured time zone can be output to the logs as a separate field. The list of time zones is derived from the IANA Time Zone database. Direct GMT offsets can also be specified.
    • Duplicate Logs: To ensure that no logs are skipped during any down time, specify the number of minutes that the NSS will send duplicate logs.
  3. Define filters as follows:

Policy Action: Use this filter to limit the logs to transactions that were either allowed or blocked. Transactions wherein the service displayed a Caution page are considered blocked transactions; if users proceeded with the transactions, they are considered allowed.

Policy Reason: Use this filter to limit the logs based on the policy that the Zscaler service applied. These are the policy reason strings that are in transaction drilldown. They indicate which policy caused a block, or if allowed, the conditions under which they were allowed, such as Allowed due to override and Internet Access cautioned. Multiple selections are allowed.

Users: Use this filter to limit the logs to specific users who generated transactions. You can search for users by user name or email address. There is no limit on the number of users that you can select. Users that are deleted after they are selected appear with a strikethrough line.

Departments: Use this filter to limit the logs to specific departments that generated transactions. You can search for departments. There is no limit on the number of departments that you can select. Departments that are deleted after they are selected appear with a strikethrough line.

Locations: Use this filter to limit the logs to specific locations from which transactions were generated. You can search for locations. There is no limit on the number of locations that you can select. Locations that are deleted after they are selected appear with a strikethrough line.

Client IP Addresses: Use this filter to limit the logs based on a client’s private IP address. You can enter:

  • An IP address (for example, 198.51.100.100)
  • A range of IP addresses (for example, 192.0.2.1-192.0.2.10)
  • An IP address with a netmask (for example, 203.0.113.0/24)

    You can enter multiple entries. Hit Enter after each entry.

Public IP Addresses: Use this filter to limit the logs based on a client’s public IP address. The internal IP address is available if traffic forwarding is forwarded to the service through a GRE or VPN tunnel or from the XFF header. If the internal IP address is not available, the value will be same as the client IP address. You can enter:

  • An IP address (for example, 198.51.100.100)
  • A range of IP addresses (for example, 192.0.2.1-192.0.2.10)
  • An IP address with a netmask (for example, 203.0.113.0/24)

    You can enter multiple entries. Hit Enter after each entry.

Direction: Use this filter to limit the logs to either inbound or outbound traffic.

User Agents: Use this filter to limit the logs to transactions associated with the user-agent string that the browser included in its GET request. Choose from the list of predefined user-agent strings or enter custom user-agent strings. Multiple selections are allowed.

Custom User Agent Strings: Use this filter to limit the logs to specific user-agent strings. A user-agent string contains browser and system information that the destination server can use to provide appropriate content.

Protocol Types: Use this filter to limit the logs to specific protocols. Supported protocols are HTTP, HTTPS and FTP. Multiple selections are allowed.

Request Methods: Use this filter to limit the logs based on the HTTP request method obtained from the client request. Multiple selections are allowed.

Response Codes: Use this filter to limit the logs based on the HTTP response code obtained from the server or generated by the ZEN. Multiple selections are allowed.

Request Sizes: Use this filter to limit the logs based on HTTP request size. Enter either a specific size or a range with a dash. By default, the service uses bytes, but you can also specify KB, MB, GB, or TB. For example: 10KB-1MB, 200. You can enter multiple entries. Hit Enter after each entry.

Response Sizes: Use this filter to limit the logs based on HTTP response size. Enter either a specific size or a range with a dash. By default, the service uses bytes, but you can also specify KB, MB, GB, or TB. For example: 10KB-1MB, 200. You can enter multiple entries. Hit Enter after each entry.

Transaction Sizes: Use this filter to limit the logs based on transaction size, which is the header and body request or response size, or the request and response size. Enter either a specific size or a range with a dash. By default, the service uses bytes, but you can also specify KB, MB, GB, or TB. For example: 10KB-1MB, 200. You can enter multiple entries. Hit Enter after each entry.

Referer URLs: Use this filter to limit the logs based on the Referer URL in the HTTP header. You can use wildcards based on the following rules:

  • *string -> Suffix matching  match URLs ending with ‘string’
  • String* -> Prefix matching  match URLs beginning with ‘string’
  • *string* -> Substring matching  match URLs containing ‘string’
  • String -> Exact matching  match URLs that are exactly ‘string’
    Multiple strings are allowed. Enter one string per line. String search is not case-sensitive.

URL Filter Type: Use this filter to limit the logs based on URLs in HTTP Requests. You can specify either a Hostname or the Full URL. You can use wildcards based on the following rules:

  • String -> Exact matching  match URLs that are exactly ‘string’
  • *string* -> Substring matching  match URLs containing ‘string’
  • String* -> Prefix matching  match URLs beginning with ‘string’
  • *string -> Suffix matching  match URLs ending with ‘string’

Host Names: Use this filter to limit the logs based on specific host names.

URL Classes: Use this filter to limit the logs to specific URL classes. Select those that you would like to include. Multiple selections are allowed.

URL Super Categories: Use this filter to limit the logs to specific URL super categories. Select those that you would like to include. Multiple selections are allowed.

URL Categories: Use this filter to limit the logs to specific URL categories. Select those that you would like to include. Multiple selections are allowed.

Server IP Addresses: Use this filter to limit the logs based on the destination server’s IP address. You can enter:

  • An IP address (for example, 198.51.100.100)
  • A range of IP addresses (for example, 192.0.2.1-192.0.2.10)
  • An IP address with a netmask (for example, 203.0.113.0/24)

    You can enter multiple entries. Hit Enter after each entry.

Cloud Application Classes: Use this filter to limit the logs to the selected cloud application classes. Multiple selections are allowed.

Cloud Applications: Use this filter to limit the logs to selected cloud applications. Multiple selections are allowed.

Malware Classes: Use this filter to limit the logs based on malware class or name. Multiple selections are allowed.

Malware Names: Use this filter to limit the logs based on specific malware or viruses that were detected. You can specify multiple malware or virus names. Use the Search function to search for either. See Referer URL for guidelines.

Advanced Threats: Use this filter to limit the logs based on the types of advanced threats that were detected. Multiple selections are allowed.

Threat Names: Use this filter to limit the logs based on specific threats that were detected. You can specify multiple threat names. Use the Search function to search for either. See Referer URL above for guidelines.

Suspicious Content: Use this filter to limit the logs based on the page risk index of a transaction. Enter either a single value or a range of values, between 0 and 100. Multiple values are allowed. Hit Enter after each entry.

File Type Categories: Use this filter to limit the logs based on the file type categories detected from the content. Multiple selections are allowed.

File Types: Use this filter to limit the logs based on the file type detected from the content. Multiple selections are allowed.

DLP Engines: Use this filter to limit the logs to transactions in which data leakage was detected based on specific DLP engines. Multiple selections are allowed.

DLP Dictionaries: Use this filter to limit the logs to transactions in which data leakage was detected based on specific DLP dictionaries. Multiple selections are allowed.

  1. Click Save and activate the change.