The intelligence of machines and the branch of computer science which aims to create it

Artificial Intelligence Journal

Subscribe to Artificial Intelligence Journal: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Artificial Intelligence Journal: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Artificial Intelligence Authors: Pat Romanski, Yeshim Deniz, Zakia Bouachraoui, Elizabeth White, William Schmarzo

Related Topics: ColdFusion on Ulitzer, Artificial Intelligence Journal

CFDJ: Article

Get a Grip on the Clicksteam

Get a Grip on the Clicksteam

Every time users visit your Web site they leave behind footprints of what they did. Usually this clickstream data ends up on your Web server's log.

Many Web site operators, especially those doing e-commerce and B2B, find this information valuable.

To take advantage of this data you can use third-party software or, in some cases, it may make sense to track it yourself. If you wish to do advanced work, such as real-time logging, personalization, real-time customer service and integration with customer data, writing your own custom logging system can get you moving in the right direction. This article demonstrates one way of doing custom logging and points out some fun things you can do with it.

The key to custom logging with ColdFusion is using the application.cfm file. As you may know, you can put an application.cfm file in any folder within your Web server directory tree. When a .cfm file is processed, it looks in the directory tree until it finds an application.cfm file, then runs the code in it. Due to this hierarchical structure, you can isolate your logging to certain areas of your Web site. You may want to start out logging a test area to see how your Web server configuration handles the load and to allow you to work out the kinks before you scale up.

In the application.cfm file you can create code that grabs CGI variables, such as the Web page, IP number and cookie data, and insert them into a database. First set up a file in your database. Hopefully your database is fairly robust, like SQL7 or Oracle. Keep in mind it'll be creating a record for each page view. In a high-volume site you could be creating tens of thousands of records each day, which can present a whole set of maintenance-type problems. Some of the concepts I'm introducing here may be more suited to low volume B2B and e-commerce sites.

The Code
To get started at a basic level, set up a table in your database that includes the following fields: ID, Date, URL and IP. Name them whatever you like, but follow your naming conventions and be careful about using reserved words (don't use the descriptive names I'm using). The ID field should be a numeric identity. The URL and IP fields should be a string or text type, keeping in mind that this could become a very large file. I use varchar in SQL7.

At this point pick an area of your Web site where you'd like to start logging and create an application.cfm file. For the beginners out there, there's no magic here. The application.cfm file is a file like any other, but it's treated differently by the ColdFusion server because it's named application.cfm. The code put in this file will run before the code in any other file in a given directory structure.

Use CGI variables returned from the Web server to pick up the Web page and visitor's IP number. A lot of information is available from CGI variables. Turn on debugging for your IP number in the Server Administrator to get a full listing of CGI variables to display after every local page load. I combine the server name, the script name and the query string to come up with a complete URL address. The following code combines them all.

<!--- Get ODBC date --->
<cfset DDate=#createodbcdate(now())#>

<!--- Get Page URL --->
<cfif #CGI.query_string# is "">
<cfset Page="HTTP://"&'#CGI.Server_Name#'&'#CGI.Script_Name#'>
<cfelse>
<cfset
Page="HTTP://"&'#CGI.Server_Name#'&'#CGI.Script_Name#'&"?"&'#CGI.query_string#'>
</cfif>

<!--- Get visitor IP --->
<cfset IIP=#CGI.Remote_Addr#>

At this point insert the variables into the database.
<cfquery name="InLog" datasource="MyDatabase" dbtype="ODBC">
INSERT INTO Log (Date, URL, IP)
Values (
<CFIF Trim(DDate)is "">Null<cfelse>#DDate#</cfif>,
<CFIF Trim(Page)is "">Null<cfelse>'#Page#'</cfif>,
<CFIF Trim(IIP)is "">Null<cfelse>'#IIP#'</cfif>)
</cfquery>
The code above represents the easy way to do this for presentation purposes. If you're serious about speed and scaling issues, use Stored Procedures and, in some cases, Triggers and Scheduled Execution.

If the Web site contains files that have the .html suffix, change the mapping so these files will be processed by the ColdFusion Server. To do this in IIS, open the Internet Service Manager and go to the properties of the Web site. Once there, click on the Home Directory tab. In the Application Settings area click on Configuration. Create a new mapping for .html with the same executable path as .cfm. There's a slight performance penalty for doing this since every .html file will now be processed by ColdFusion.

Big Brother Is Watching You
As people enter your site, you now have real-time clickstream data available. Creating live reports for marketing, management and Webmasters is a simple task.

Now you can start working with personalization techniques. If you have a B2B site in which customers log in with a name and password and/or have an identifying cookie set, you can send e-mail to the salespeople in charge of the account and allow them to view the user's click path. You can also have mechanisms on certain pages that will alert staff to the page hit, telling them who the users are and where they came from. Other more complex applications include having customer service staff watch every unique visit on an e-commerce site. If your technology can handle it, they can attempt to chat with the customer in an effort to obtain a higher level of customer service. You can also change links and add banner ads and other graphics dynamically, based on a customer's prior click path, in an attempt to enhance his or her visit and present customized sales opportunities.

Some of these things don't necessarily require direct access to clickstream data, but once you implement a custom logging scheme, opportunities for building artificial intelligence into your Web site become more apparent. Most of these require adding conditional logic to the application.cfm file or using Triggers on the database side and sometimes integrating existing customer data.

I've provided a simple example of how to get started with custom logging and some big picture ideas on where it can take you. As you can see, having full control of your Web site's clickstream can open up a brave new world of opportunity.

More Stories By Lee Surma

Lee Surma is a senior manager of Internet technologies at Public Radio International, a Minneapolis-based distributor of public radio programming. He has five years of experience in various Web-database initiatives.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.