NSS Feed Output Format


NSS Feed Output Format

This article provides guidelines and information about the fields that you can include in the NSS output for web logs and for firewall logs. Enter the desired fields in the Field Output Format field.

Guidelines

Following are some guidelines to get you started. Note that the applicability of the recommendations may vary based on the capabilities of your SIEM.

  • An NSS can include up to 50 fields per transaction.
  • The fields must be in the order you want them to appear in the output. Consider using name-value pairs if your SIEM can support it. Some SIEMs, such as Splunk, can automatically parse name-value formats and auto-detect field names regardless of the order.
  • The Zscaler service hex encodes all non-printable ASCII characters that are in URLs when it sends the logs to the NSS. Any URL character that is less than and equal to 0x20 or equal to and above 0x7F will be encoded as %HH. This ensures that your SIEM will be able to parse the URLs in case they contain control characters. For example, a \n character in a URL is encoded as 0A, and a space is encoded as %20.
  • Use tabs as field delimiters. Avoid using commas and spaces because they could be in some output values. For example, department names might contain spaces or URLs might contain commas. If you must use commas or other characters as delimiters, enter those characters in the Feed Escape Character field when you define an NSS feed. The service will encode the characters that you enter in this field. For example, if you enter a comma, the service will encode it as %2C.
  • The following special characters generate control outputs:
    \t = Tab
    \n = Newline
    \r = Carriage Return
  • You do not need to add \n after the last field to indicate that it is the last entry. Zscaler automatically adds a blank line (line feed) after the last field.
  • The field names specified will be substituted by the corresponding value when NSS sends logs to the SIEM.
    • %s{<fieldname>} will display the string name
      Example: %s{time}
    • %d{<fieldname>} will print a number in decimal format
      Example: %d{yyyy}
    • %x{<fieldname>} will print a number in hexadecimal (base-16) format
      Example: %x{recordid}
      Following are some examples:
      • "%s{time}","%s{login}","%s{proto}"\n will result in a log entry that looks like: “Mon Sep 10 10:50:46 2012”,”jdoe@example.com”, “HTTP”
      • %s{time}\t%s{login}\t%s{proto}\n will result in a log entry that looks like: 
        Mon Sep 10 10:50:46 2012jdoe@example.comHTTP
  • Field names are case-sensitive.
  • Special characters must be escaped with \{} or % . For example: %s{login}: \{%s{hostname}\} will generate a log entry similar to the following: user@example.com: {www.zscaler.com}
  • You can optionally include static text strings in the output. For example: %s{time} GMT will produce: Mon Sep 10 10:50:46 2012 GMT
  • You can also specify the padding if you want constant width strings. For instance:
    • %d{hh}:%d{mm}:%d{ss} will produce 21:5:0
    • %02d{hh}:%02d{mm}:%02d{ss} will produce 21:05:00
  • % must be escaped with another %
  • Constant width with leading zeroes can be provided:
    • %d{hh}:%d{mm}:%d{ss} will produce 21:5:0
    • %02d{hh}:%02d{mm}:%02d{ss} will produce 21:05:00
  • You can break up a long output format into multiple lines for readability. 
    For example, the following format string: 
    %s{time}\t%s{proto}\t%s{url}\t%s{login}\t%s{dept}\t%s{urlcat}\t%s{urlsupercat}\t%s{urlclass}\t%s{location}\t%s{ua}\n
    Can be specified as:
    %s{time}\t%s{proto}\t%s{url}\t
    %s{login}\t%s{dept}\t
    %s{urlcat}\t%s{urlsupercat}\t%s{urlclass}\t
    %s{location}\t%s{ua}\n

Web Log Fields

The following fields display information about the time of a transaction.

Field Description Example

%s{time}

Time and date of the transaction. This excludes the time zone.    

Thu Sep 6 22:55:48 2012

%s{tz}    

The time zone. This is the same as the time zone you specified when you configured the NSS feed.

GMT

%d{ss}    

Seconds (0-59)

48

%d{mm}

Minutes (0-59)

55

%d{hh}

Hours (0-23)

22

%d{dd}

Day of the month (1-31)

16

%d{mth}

Month of the year

9

%s{mon}

Name of the month

Jan

%d{yyy}

Year

2012

%s{day}

Day of the week

Mon


The following fields display information about the action that was taken on a transaction.

Field Description Example

%s{action}

The action that the service took on the transaction: Allowed or Blocked.    

Allowed

%s{reason}

The action that the service took and the policy that was applied, if the transaction was blocked.    

Virus/Spyware/Malware Blocked

%s{bwthrottle}

Indicates whether the transaction was throttled due to a configured Bandwidth policy.   

yes


The following fields display information about the destination of a transaction.

Field Description Example

%s{app}

The name of the application that was accessed.

Google Mail

%s{module}

The web application class of the application that was accessed.

General Browsing

%s{host}

The destination host name.

mail.google.com

%s{ehost}

The encoded version of the destination host name.

 

%s{proto}

The protocol type of the transaction.

HTTP

%s{sip}

The destination server IP address. Displays 0.0.0.0 if the request was blocked.

1.1.1.1

%s{url}

The destination URL. It excludes the protocol identifier, such as http:// or https://

www.trythisencodeurl.com/index

%s{eurl}

The encoded version of the destination URL.

www.trythisencodeurl.com/index%1A%09

%s{urlcat}

The category of the destination URL

Professional Services

%s{urlclass}

The class of the destination URL.

Business Use

%s{urlsupercat}

The super category of the destination URL.    

Business and Economy


The following fields display information about the content of a transaction.

Field Description Example

%s{bamd5}

The MD5 digest of the file that was sent for analysis to the Sandbox engine.

196a3d797bfee07fe4596b69f4ce1141

%s{dlpdict}

The DLP dictionaries that were matched, if any.

Credit Card Number

%s{dlpeng}

The DLP engine that was matched, if any.

PCI

%s{fileclass}

The super category of the field type.

Microsoft Office Documents

%s{filetype}

The type of file associated with the transaction.

Microsoft PowerPoint

%s{filename}

The name of the file associated with the transaction

nssfeed.txt

%s{malwarecat}

The category of malware that was detected in the transaction, if any. Also indicates if a file was submitted to the Sandbox engine for analysis and the result of the analysis.

Boot Virus

Submitted to BA server 

%s{malwareclass}

The class of malware that was detected in the transaction, if any.

Virus

%d{riskscore}

The Page Risk score of the destination URL. The service computes risk for each page by weighing several factors, including page locations, reputation of destination, and content that may look suspicious. The range is 0 - 100, from the lowest to the highest risk.    

10

%s{rulelabel}

The rule name

Bandwidth_1

%s{ruletype}

The policy type

URL & Cloud App Control Policy

%s{threatname}

The name of the virus that the service detected in the transaction, if any.

EICAR Test File


The following fields display information about the source of a transaction.

Field Description Example

%s{cintip}

The client Internet (NATted Public) IP address. This is different from the cip value if the internal IP address is visible. Otherwise it is the same as the cip value.    

203.0.113.5

%s{cip}    

The IP address of the user. This can be the internal IP address if it is visible; for example, traffic sent through a GRE tunnel or an internal IP address indicated using XFF. Otherwise, same as ‘cintip’    

203.0.113.5, 192.168.2.200

%s{location}

The gateway location or sublocation of the source.    

Headquarters

%s{mobappcat}

The mobile app type, if any.

Games, 3rd Party Communication

%s{mobappname}

The name of the mobile app, if any.

Pinterest, Facebook

%s{referer} 

The HTTP referer URL.

www.google.com

%s {ereferer} The encoded version of the HTTP referer URL.  

%s{ua} 

The user-agent string that the browser included in its GET request. The user-agent string contains browser and system information that the destination server can use to provide appropriate content.    

Firefox/15.0


The following fields display information about the user of a transaction.

Field Description Example

%s{dept}

The department of the user.    

Sales

%s{login}

The user's login name, in email address format. 

jdoe@safemarch.com

%s{ologin}

If obfuscation is enabled, this displays a random string. If obfuscation is disabled, then this is blank.    

4094304256

The following fields display information about the transaction.

Field Description Example

%d{ctime}

Total time, in milliseconds, from when the browser made the first request to the cloud and returned all the content to the browser.    

500

%d{recordid}

Unique record identifier for each log.    

 

%d{reqdatasize}

Size, in bytes, of the HTTP Request payload, excluding the headers.    

1000

%d{reqhdrsize}

Size, in bytes, of the HTTP Request header.   

300

%s{reqmethod}

The HTTP request method.    

get

%d{reqsize}

Request size in bytes.   

1300

%s{reqversion}    

The HTTP request version.     

1.1

%s{respcode}

The HTTP response code sent to the client. The service generates a "403-Forbidden" response for blocked transactions.    

403-Forbidden

%d{respdatasize}

Size, in bytes, of the HTTP Response payload, excluding the headers.    

10000

%d{resphdrsize}

Size, in bytes, of the HTTP Response header.    

500

%d{respsize}

Total size, in bytes, of the HTTP response, includes the header and payload.     

10500

%s{respversion}

HTTP response version.   

1.0

%d{totalsize}

Total size, in bytes, of the HTTP transaction; sum of the total request size and total response size.   

11800

%s{eedone}

Indicates if the characters specified in the Feed Escape Character field of the NSS Feed Configuration page were hex encoded.

YES


The following fields display NSS meta-information.

Field Description Example

%s{nsssvcip}

The service IP address of the NSS. Useful for syslog-format logs that require the origin host IP address to be specified.    

10.10.102.300

%s{productversion} 

The current version of the product. Useful for SIEMs whose format requires the product internal version to be sent in the log output.    

5.0.902.95524_04

Firewall Log Fields

The following fields display information about the time of a transaction.

Field  Description Example

%s{time}

Time and date of the transaction. This excludes the time zone.    

Tue Jul 7 22:55:48 2015

%s{tz}

The time zone. This is the same as the time zone you specified when you configured the NSS feed.

GMT

%02d{ss}

 Seconds (0-59)

45

%02d{mm}

 Minutes (0-59)

30

%02d{hh}

Hours of timestamp (0-12)

1

%02d{dd}

Day of timestamp (1-31)

18

%02d{mth}

Name of the month

October

%04d{yyyy}

 Year 

2015


The following fields display information about the action that was taken on the transaction.

Field  Description Example

%s{rulelabel}

The name of the the rule that was applied to the transaction.

Default firewall filtering rule

%s{action}

The action that the service took on the transaction: Allowed or Blocked. 

Blocked

%s{dnat}

Indicates if the destination NAT policy was applied.

Yes


The following fields display information about the sessions.

Field  Description Example

%d{avgduration}

Average session duration, in milliseconds, if the sessions were aggregated.

600,000

%d{duration}

Session or request duration in seconds.

600

%d{durationms}

Session or request duration in milliseconds.

600,000

%d{numsessions}

Number of sessions that were aggregated.

5


The following fields display information about the client.

Field  Description Example

%s{csip}

Client source IP address. For aggregated sessions, this is the client source IP address of the last session in the aggregate

192.0.2.10

%d{csport}

Client source port. For aggregated sessions, this is the Client source port of the last session in the aggregate.

22

%s{cdip}

Client destination IP address. For aggregated sessions, this is the client destination IP address of the last session in the aggregate.

198.51.100.54

%d{cdport}

Client destination port. For aggregated sessions, this is the Client destination port of the last session in the aggregate.

22

%s{tsip}

Tunnel IP address of the client (source). For aggregated sessions, this is the client's tunnel IP address corresponding to the last session in the aggregate.

192.0.2.15

%d{tsport}

Tunnel port on the client side. For aggregated sessions, this is the client's tunnel port corresponding to the last session in the aggregate. 

22

%s{location}

Name of the location from which the session was initiated.

Headquarters

%s{ttype}

Traffic forwarding methods that was used to send the traffic to the firewall

L2 tunnel


The following fields display information about the server.

Field  Description Example

%d{sdport}

Server destination port. For aggregated sessions, this is the server destination IP address of the last session in the aggregate.

192.0.2.100

%s{ssip}

Server source IP address. For aggregated sessions, this is the server source IP address of the last session in the aggregate.

198.51.100.100

%d{ssport}

Server source port. For aggregated sessions, this is the server source port of the last session in the aggregate.

22

%s{ipcat}

URL category that corresponds to the server IP address. 

Finance


The following fields display information about the transaction.

Field  Description Example

%d{recordid}

 Record ID 

 

%ld{inbytes}

The number of bytes sent from the server to the client

10000

%s{nwapp}

The network application that was accessed

Skype

%s{ipproto}

 The IP protocol type

TCP

%s{destcountry}

 The abbreviated code of the country of the destination IP address

USA

%s{nwsvc}

 The network service that was used

HTTP

%s{eedone}

Indicates if the character specified in the Feed Escape Character field of the NSS Feed Configuration page were hex encoded. 

YES


The following fields display information about the user.

Field Description Example

%s{login}
     

The user's login  name, in email address format

jdoe@safemarch.com

%s{dept}
     

The department of the user

Sales

DNS Feed Output Format

The following fields display information about the time of a transaction.

Field  Description Example

%s{time}

Time and date of the transaction. This excludes the time zone.    

Tue Jul 7 22:55:48 2015

%02d{ss}

 Seconds (0-59)

45

%02d{mm}

 Minutes (0-59)

30

%02d{hh}

Hours of timestamp (0-12)

1

%02d{dd}

Day of timestamp (1-31)

18

%02d{mth}

Name of the month

October

%04d{yyyy}

 Year 

2015


The following fields display information about the action that was taken on the transaction.

Field Description Example

%s{rulelabel}

The name of the the rule that was applied to the transaction.

 

%s{action}

The action that the Zscaler performed on the DNS request or response. 

REQ_ALLOW
RES_BLOC

%s{res}

 The response from the DNS server.

 


The following fields display information about the user of a transaction.

Field Description Example

%s{login}

 The user's login name, in email address format.

jdoe@safemarch.com

%s{dept}

The department of the user.

Sales


The following fields display information about the transaction.

Field Description Example

%s{cip}

The IP address of the user. This can be the internal IP address if it is visible; for example, traffic sent through a GRE tunnel or an internal IP address indicated using XFF. Otherwise, it is the client Internet (NATted Public) IP address.

203.0.113.5

%d{durationms}

 Duration of the DNS request in milli-seconds.

 

%s{sip}

 Server IP address of the request.

192.168.2.200

%d{recordid}

Unique record identifier for each log.

 

%s{location}

The gateway location or sub-location of the source.

Headquarters

%s{req}

The Fully Qualified Domain Name (FQDN) in the DNS request.

mail.safemarch.com

%s{domcat}

 URL Category of the FQDN in the DNS request.

Professional Services

%s{reqtype}

 The DNS record being requested.

A record

%d{sport}

 Server port of the request.

 

%s{eedone}

Indicates if the characters specified in the Feed Escape Character field of the NSS Configuration page were hex encoded.