IIS Log file formats overview

Written by Steen Jakobsen
27-03-2017

This article discusses the different log file formats (W3c Extended Log file format, IIS Log file format, NCSA Common Log file format and ODBC Logging) the Log file format and file naming syntax.

 

This article also illustrates how to enable, configure and customize logging in Internet Information Services and where to look for log files.

Ontolica Search Intelligence gathers information from IIS log files to create data that can be used to create various reports that will help in analyzing various statistics such as popularity of recourses, people who have visited the site, amount of data transferred and search engine usage to name a few. It is important to record the IIS log information in a proper format so that it can be read and analyzed by the Ontolica Search Intelligence correctly. This article will give a brief overview of different log file formats supported by IIS, log file naming syntax, log file storage location, enabling/configuring logging and lists the log file formats supported by Ontolica Search Intelligence.

Internet Information Server supports different log file formats where you can collect information about client requests. Whenever an HTTP transaction occurs, IIS writes these requests for sites and services into a log file in selected log file format. By analyzing these log files manually or programmatically you can find out information such as server errors, client activity, how much data has been transferred/received, who has visited your site, what has been viewed, when it was last viewed etc.

Logs file formats supported by IIS.

  • W3C (World Wide Web Consortium) Extended log file format – This is the default log file format used by IIS. Its uses ASCII text format and the time are recorded as UTC. This is the only format where you can customize the properties there by you can limit the size of log files and obtain the detailed information. The properties written in the log files are separated by using spaces.
  • IIS (Microsoft Internet Information Services) log file format – This format also uses ASCII text format and uses fixed number of properties. IIS log file format is used when you don’t need detailed information from the logs; it logs more information than NSCA common format but less than W3C format. It is a comma separated file and uses the local time.
  • NCSA (National Center for Supercomputing Applications) log file format – This format logs only the basic information. Similar to IIS log file format it uses fixed number of properties. It records the time using the local time and properties are separated by spaces. Note that NCSA log file format does not support FTP sites. Since the entries are small with this format, the storage space required for logging is comparatively less compared to other formats.
  • Centralized Binary Logging – Centralized binary logging is used when multiple web sites running on a server to write binary, unformatted log data to a single log file. Each web server running IIS creates one log file for all sites on that server. The IIS writes log files in binary format and uses a single file there by making it memory efficient. This type of logging is not supported at web site level.
  • ODBC log file format – This method is used when you want to log access information directly to a database. Enabling ODBC logging will disable the kernel-mode cache so this may affect the server performance. Only supported at site level.

Following table lists sample log entries for different log format files. A hyphen (-) appeared in this samples shows that there is no information for that field.

Format

Sample

W3C

#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2009-06-11 05:12:03
#Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
2009-06-11 05:12:02 W3SVC1893743816 192.168.1.109 GET / – 4677 – 192.168.1.109 Mozilla/4.0 (compatible;+MSIE+4.01;+Windows+NT;+MS+Search+5.0+Robot) 401 2 2148074254 2009-06-11 05:12:02 W3SVC1893743816 192.168.1.109 GET / – 4677 – 192.168.1.109 Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+NT;+MS+Search+5.0+Robot) 401 2 2148074254

IIS

192.168.1.109, -, 6/10/2009, 10:11:59, W3SVC1893743816, SPUTNIK01, 192.168.1.109, 0, 261, 1913, 401, 2148074254, GET, /, -, 192.168.1.109, -, 6/10/2009, 10:11:59, W3SVC1893743816, SPUTNIK01, 192.168.1.109, 15, 363, 2113, 401, 0, GET, /, -, 192.168.1.109, NT AUTHORITY\LOCAL SERVICE, 6/10/2009, 10:11:59, W3SVC1893743816, SPUTNIK01, 192.168.1.109, 46, 379, 336, 200, 0, GET, /, -, 192.168.1.109, -, 6/10/2009, 10:11:59, W3SVC1893743816, SPUTNIK01, 192.168.1.109, 0, 336, 1889, 401, 2148074254, POST, /_vti_bin/sitedata.asmx, -,

NCSA

192.168.1.109 – - [08/Jun/2009:12:11:14 +0200] “GET / HTTP/1.0″ 401 1913 192.168.1.109 – NT+AUTHORITY\LOCAL+SERVICE [08/Jun/2009:12:11:14 +0200] “GET / HTTP/1.0″ 200 336 192.168.1.109 – - [08/Jun/2009:12:11:14 +0200] “POST /_vti_bin/sitedata.asmx HTTP/1.1″ 401 1889 192.168.1.109 – NT+AUTHORITY\LOCAL+SERVICE [08/Jun/2009:12:11:14 +0200] “POST /_vti_bin/sitedata.asmx HTTP/1.1″ 200 1331

 

Following table lists the available fields in different log formats.

Fields

Description

W3C

NCSA

IIS

Date The date on which the activity occurred.

ü

ü

ü

Time The time, in coordinated universal (UTC), at which the activity occurred.

ü

ü

ü

Service Name and Instance number The internet service name and instance number that was running on the client.

ü

X

ü

Server Name The name of the server on which the logs file entry was generated.

ü

X

ü

Server IP Address The IP address of the server on which the logs file entry was generated.

ü

X

ü

Method(Request type) The requested verb, for example, a GET method.

ü

ü

ü

URI Stem(Target of Operation) The target of the verb, for example, Default.htm.

ü

X

ü

URI Query The query, if any, that the client was trying to perform. A universal Resource Identifier (URI) query is necessary only for dynamic pages.

ü

X

X

Server port The server port number that is configured for the service

ü

X

X

User Name The name of the authenticated user that accessed the server. Anonymous users are indicated by a hyphen.

ü

ü

ü

Client IP Address The IP address of the client that made the request.

ü

X

ü

Protocol Version The HTTP protocol version that the client used.

ü

ü

X

User Agent The browser type that the client used.

ü

X

X

Cookie The content of the cookie sent or received, if any.

ü

X

X

Referrer The site that the user last visited. This site provided a link to the current site.

ü

X

X

Host The host header name, if any.

ü

ü

X

HTTP Status(Service status code) The HTTP status code. A value of 200 indicates that the request was fulfilled successfully.

ü

ü

ü

Protocol Sub status The sub status error code.

ü

X

X

Win32 Status The windows status code. A value of 0 (zero) indicates that the request was fulfilled successfully.

ü

X

ü

Bytes Sent(Server bytes sent) The number of bytes sent by the server.

ü

ü

ü

Bytes Received The number of bytes received and processed by the server.

ü

X

X

Time Taken The length of time that the action took, in milliseconds.

ü

X

ü

Client bytes sent The number of bytes sent by the client

X

X

ü

Parameters The parameters that are passed to a script.

X

X

ü

 

Log files location:

By default the IIS log files are stored under following folder:

  • IIS 6.0 : % system32%\LogFiles\W3SVCN
  • IIS 7.0 : %SystemDrive%\Inetpub\Logs\LogFiles \ W3SVCN

 

Where W3SVCN is the subdirectory for a particular site and N is the index number of the service. It is possible to change this default location to the one you want to use while enabling the logging in IIS.

 

How to enable logging?

Enabling logging is a simple process. You can enable logging at the server level or at site level. By enabling at server level will enable logging for all of the configured web sites on that server in a single log file. Enabling at site level will create a separate log files for each web site. To enable the IIS loggings follow these steps.

  1. Start Internet Information Service Manger.
  2. Navigate to your server/website, Right click and select properties
  3. Go to Web Site tab and select Enable logging check box
  4. Choose the type of format you want to use.
  5. Click Properties to customize the settings you want.

Note that by default, W3C Extended log file format uses midnight coordinated universal time (Greenwich Mean Time). All other log file formats uses the midnight local time format. To use local midnight time for W3C extended log format, click to select the Use local time for file naming and rollover check box.

 

IIS Log File name conventions:

IIS uses the following file name syntax for files using UTF-8 encoding. If you are using UTF-8 encoding then these log files will have an “u_” prefix. For example u_extendnn.log is the log file according to file size for W3C extended log format using UTF-8 encoding. Log file name indicates how log files are created, what time schedule it was created, is it a fixed size files or unlimited size and the type of encoding used.

Log Interval W3C Extended Log file format NCSA Common Log file format IIS Log file format
By File size extendnn.log ncsann.log Inetsvnn.log
Hourly exyymmddhh.log ncyymmddhh.log Inyymmddhh.log
Daily exyymmdd.log ncyymmdd.log Inyymmdd.log
Weekly exyymmww.log ncyymmww.log Inyymmww.log
Monthly exyymm.log ncyymm.log Inyymm.log

 

IIS Log file formats supported by Ontolica Search Intelligence

Ontolica Search Intelligence supports importing and analyzing data from the IIS log files created using following file formats.

  • W3C Extended Log file format
  • IIS Log file format
  • NCSA Common Log file format

Note: Currently Ontolica Search Intelligence only supports log files created using Unicode/ASCII encoding. Since IIS does not support the UTF-8 format for FTP sites log files, it is not supported by OSI either. By default IIS 7.0 uses UTF-8 encoding so you need to disable the UTF-8 encoding.

How to disable UTF-8 Encoding?

  1. Go to IIS Manager, right-click on the local computer, and then select Properties.

 

  1. In UTF-8 Logging, unselect the Encode Web logs in UTF-8 check box, and then click OK.

 
ONTOLICA by SURFRAY | (C) 2017, 37229539