Monitoring the number of views of an important corporate document is a key need for any company who is aiming for better resource management and company-wide alignment. This is especially important in a migration project in order to define what content to migrate or in a document maintenance process to prioritize assignments. SAP BusinessObjects Auditing is able to register when documents are viewed or retrieved but it has certain limitations, so most of the users activity is lost. This article explains an alternative method to collect the number of views of any SAP BusinessObjects file type or web page that sits on our web server so they can complement our current auditing reports.
The Scenario
Our case was a customer running SAP BusinessObjects 3.1. one who wanted to migrate to SAP BI4, and the first question they asked for is about usage load per document type. The documents they wanted to monitor were:
Web Intelligence and Desktop Intelligence documents
Web Intelligence and Desktop Intelligence instances
PDF or Excel instances
SAP Dashboards (Xcelsius) (flash format .swf)
Agnostic documents (pdf, xls, ppt, doc)
Open document calls to any of the above
Explorer Information Spaces
HTML ad-hoc portals located in Tomcat Server
The SAP BusinessObjects Audit database contains part of this information, but it has strong limitations:
If a document is deleted the historical hits go away
If an instance disappears – which is very common, because instances are automatically deleted when the limit is reached – historical hits go away
It does not register hits of any other kind (such as items 3-8 from the list indicated above)
Given the need for a complete set of information, we decided to explore the HTTP traffic logs coming from the SAP BusinessObjects Tomcat Server to see if we can get this missing information. Here is the result.
Tomcat Logs
The files containing the HTTP traffic are generated every day in the following folder: $TOMCAT_HOME/logs. This generation does not occur by default, you can enable it by uncommenting the following sentence in the server.xml file located at $TOMCAT_HOME/conf :
<Valve className="org.apache.catalina.valves.FastCommonAccessLog Valve"..directory="logs" prefix="localhost_access_log." suffix=".txt" pattern="combined" resolveHosts="true"/>
Immediately the files start generating a lot of information. Find below an example of a “View” hit on a WebI report:
10.60.89.150 - - [04/Apr/2013:08:19:18 +0200] "GET /AnalyticalReporting/WebiView.do?cafWebSesInit=true&appKind=InfoView&service=/InfoViewApp/common/appService.do&loc =en&pvl=nl_NL&ctx=standalone&actId=533&objIds=444645&containerId=444634&pref =maxOpageU%89.150D10%89.150BmaxOpageUt%89.150D200%89.150BmaxOpageC%89.150D10%89.150Btz%89.150DEurope %89.150FBerlin%89.150BmUnit%89.150Dinch%04BshowFilters%04Dtrue%04BsmtpFrom%04Dtrue%04Bprompt ForUnsavedData%04Dtrue%04B HTTP/04" 04 04 "2013 "Mozilla/2013 (compatible; MSIE2013; Windows NT2013; Trident/2013; SLCC2; .NET CLR2013.2013; .NET CLR2013.2013; .NET CLR2013.08:19; .NET4.08:19C; .NET4.08:19E; InfoPath.08:19; MS-RTC LM 08:19)"
With a quick check we detect there are very interesting fields for us:
User Name: Remote host name or IP address (if resolveHosts is false)
Fecha y hora
First line of the request (method and request URI)
HTTP status code of the response
In the last 2 fields is where most of the information is located. All the hits to different objects and their object Id or calls to http pages are registered here, as well as user names occasionally. If you wish to customize the format of the file there are many possibilities, just check from the Tomcat's vendor’s page.
So now we have the data we need. Next challenge is how to pick up the hits and transform them in order to fit in our auditing reports.
Data Transformation
The stages for data transformation & enrichment proposed here are as follows;
Consolidation of the source and hits filtering
Data enrichment & normalization:
Storage of deleted files and instances information
Assignment of the hit to the document – not the instance
Enrichment of this info with attributes like Kind, Name, Owner
Replacement of the IP with the User Name
Incorporation of other hits like Client Tools coming from the BO Audit database
Final hits filtering: Exclusion of hits from Administrators or instance owners
For a full sequence of the transformation you can check the following diagram (Fig.1.):
Fig.1 Flow to obtain the number of Views of any SAP BusinessObjects document
The tricky part here is the “hits filtering” (initial one) which is mainly based on experience. Whatever is the hit you want to visualize, just reproduce it, and right after open the log file and search for it to see how it looks like and determine the perfect filtering pattern.
In our specific case a single SAP BusinessObjects Universe was used and for any new hit type we wanted to monitor a predefined condition was added with an OR statement. Below a few examples based on the source fields Method and HTTP:
Open document calls:
HTTP is equal to “-”
Method contains
Web Intelligence hits:
Method contains {“/WebiModify.do?”, “WebiView.do?”}
Desktop Intelligence hits:
Method contains {“FullClientModify.do?”, “FullClientView.do?”}
Agnostic documents and hyperlinks:
Method contains {“PlatformServices/content/view.do?”, “/hyperlink/view.do?”}
Once determined all the filtering patterns for the hits we need to monitor, we can enrich this information with data coming from the CMS System database as well as the CMS Audit Database. The result of all these data transformations is a single file containing hit information by:
Day & Time
Usuario
Object Code & its attributes (kind, name, owner)
Data monitoring
The resulting data set can be monitored in any BI visualization tool, find below a sample in Web Intelligence (Fig.2):
Fig.2 Number of hits by time, user and report type
Resumen
In this exercise of data exploitation, transformation and monitoring, we have improved the auditing capabilities of SAP BusinessObjects due to the customer requirement of “not missing a hit”.
The benefit of this method is to provide hits that were impossible to detect previously:
Deleted content
Agnostic documents like pdf, xls, ppt, doc, flash
Explorer Information Spaces
Open Document calls
HTML portals and any other web page accesses
This will allow Administrators and IT managers to have a flavour of what content to target and prioritize in migrations or maintenance tasks, which will improve drastically their efficiency.
If you have questions about this article, or if you want to share your experience or tips, please feel free to leave a comment.