Homemade Page Timing

---

I started a new job a few months ago and inherited a mess. Don't believe me? Friday I found

<cfif isDefined("form.loginSubmit") and true>

in the login script.

I actually found

<cfsetting requestTimeOut = "240000" >

I mean seriously? 66 Hours???!

Anyway, the job consists of doing some pretty heavy data analysis of millions of records and hundreds of tables/views. Needless to say the queries don't run so hot. Some of the pages don't even run at all.

We're using CF9 enterprise; however, I don't have access to any of the logs or server monitor information so I had to make my own. This may also be helpful to anyone using CF Standard edition without server monitoring.

onRequest to the rescue! And in true CF fashion, it's pretty simple to do.

The initial requirement was to track what reports were being used and how often. That's easy enough. While I was doing that, I wondered if it would be possible to time how long the page takes to run. With several reports timing out it would be good to know these things. I had never tried doing any processing after including the targetPage, but I didn't see any reason why it wouldn't work.

<cfset startTimer = getTickCount()>
<cfinclude template = "#arguments.targetPage#">
<cfset timeSpent = getTickCount() - variables.startTimer>

After getting the timeSpent I saved the user, date, file name, and timeSpent to a new table. This afforded me the opportunity to get some analysis on our analysis. How ironic. I was able to create two quick views. The first is simply the data I saved highlighting the longer running reports by color. The other was the fastest, average, and slowest processing times and a hit count of the report.

Not to my surprise I found several pages that take 7+ minutes to process... I have my work cut out for me...

---

You're such a dork!

---

...is what my wife said to me Saturday at Best Buy.
She might be right, but what else was I supposed to do? I felt it was my duty to educate the general public and entertain myself in a childish yet strangely satisfying way -- I kind of felt like Cereal Killer being generally mischievious (only not as cool).

The guy at the Apple table wasn’t looking so I made all the iPads look like:



and continued my shopping. I glanced over at the display on my way out and the screen was still up on all 4. It’s a shame I couldn’t set the homepage.

---

OnRequest required to access page variables scope in onError.

---

I was playing around with onError the other day getting ready to upgrade some legacy code I just inherited. Because I was only concerned with onError I didn't bother to add onRequest during my testing. This is the first time I've never had onRequest in my application.cfc so I didn't realize there was such a dependence between onError and onRequest until then. Ben Nadel blogged about the same thing quite a while ago when he was missing the root cause structure after deleting the onRequest method.

I hadn't gotten to dumping the actual error contents yet, I was more concerned with the variables scope... which was void of the data I was expecting it to contain. I knew right away what the problem was and it makes total sense, but having never experienced it before, gave me one of those "no shit, Sherlock" moments.

The problem is that without <cfinclude template = "#arguments.targetPage#"> the page's variable scope is OUTSIDE of application.cfc so dumping #variables# only gives you the variables you set in your application.cfc. The cfinclude effectively brings the page processing into the cfc giving onError access to those variables.

If you're one of those people still using cferror, having direct access to the variable scope is worth the change by itself.

---

ColdFusion likes lazy programmers

---

I don’t usually like to alias my code. Even when I write SQL I like to use the full field name, schema.table.column. I only use aliases when I’m in a sub query and need to reference the main query; then it’s required. I do the same thing in ColdFusion, I don't usually assign deep variables to a reference (no I don't really think it's lazy, just my preference).


Recently I was working with xml documents uploaded by users. For some reason my looping through my collection was timing out. My test document has 12,804 child elements from the root. At first I thought it was the processing I was doing within the loop. I kept removing sections of code trying to find out where the problem was. Finally I was left with a single cfoutput of a single element and my code was still timing out.


I’m not sure why I decided to try, because it isn't something I would normally do, but I assigned the parsedXML to a reference. To my surprise this solved my timeout problem. It solved it so well... my head exploded.


Here is a simple test to show the contrast.
The first loop uses the full variable syntax:
variables.parsedXML.XMLrootElement.XMLChildElement[variables.i].id.xmlText
The second loop uses the reference:
variables.XMLByRef[variables.i].id.xmlText

<cfsetting requesttimeout="960" /> 
<cfset variables.parsedXMLLen = arrayLen(variables.parsedXML.XMLrootElement.XMLChildElement) > 
<cfset variables.startXMLLongLoop = getTickCount() > 

#variables.parsedXML.XMLrootElement.XMLChildElement[variables.i].id.xmlText#


<cfset variables.stopXMLLongLoop = getTickCount() >

<cfset variables.XMLByRef = variables.parsedXML.XMLrootElement.XMLChildElement > 
<cfset variables.startXMLShortLoop = getTickCount() >

#variables.XMLByRef[variables.i].id.xmlText#

<cfset variables.stopXMLShortLoop = getTickCount() >

Fully qualified: #variables.stopXMLLongLoop-variables.startXMLLongLoop# miliseconds to loop #variables.parsedXMLLen# times
Referenced: #variables.stopXMLShortLoop-variables.startXMLShortLoop# miliseconds to loop #variables.parsedXMLLen# times 

The result:
Fully qualified: 537765 miliseconds to loop 12804 times
Referenced: 250 miliseconds to loop 12804 times

The first loop using #variables.parsedXML.XMLrootElement.XMLChildElement[variables.i].id.xmlText# took almost 9 minutes to run!!
The second loop using #variables.XMLByRef[variables.i].id.xmlText# only took a quarter second??!! WTF just happened here? Anyone know why using a reference variable is faster than using the fully qualified variable?

Thanks!

---

ColdFusion and ORA-01000: Too many open cursors

---

Today one of our sites started throwing an "ORA-01000: Too many open cursors" error. I have exactly 0 control over our server environment nor do I have access to the ColdFusion administrator. I have even less influence on our Oracle environment. We noticed the problem just before our daily scrum meeting so if nothing else, it got me out of that. Of course the first thing our sys admin did was contact our Oracle DBAs to see if they were having any problems (let the finger pointing begin, well actually I started it). They verified our data source connections and sent me a copy of the log files, which I already know weren't going to help much.

In my research, one of the Google results I came across made me chuckle. It was a Macromedia (now Archived Adobe) TechNote blaming the problem on Oracle. It said to change the OPEN_CURSORS and CLOSE_CACHED_OPEN_CURSORS settings.

It would appear that the Max Pooled Statements setting was increased in CF9, which we just upgraded too. If ColdFusion tries to keep more cursors open than Oracle will allow this error will occur. ColdFusion does this with queries using cfqueryParam and cfstoredproc to improve performance.

From Livedocs:

Max Pooled Statements enables reuse of prepared statements (that is, stored procedures and queries that use the cfqueryparam tag). Although you tune this setting based on your application, start by setting it to the sum of the following:
Unique cfquery tags that use the cfqueryparam tag
Unique cfstoredproc tags

Don't mistake Max Pooled Statements for the Maximum number of cached Queries under Server Settings > Caching, they're not the same. Max Pooled Statements can be found in the advanced settings of the data source (but I think only for SQL Server and the native Oracle driver).

The problem is, for some reason our data source was setup to connect using ODBC rather than the native Oracle drivers that come with ColdFusion Enterprise. There IS NO Max Pooled Statements setting for ODBC connections under advanced settings. Does that mean there's a similar setting in the background somewhere for miscellaneous databases? Was this setting increased in CF9? I don't know. I looked but can't find anything.

We finally got the problem solved by recreating the data source using the Oracle drivers and lowering the Max Pooled Statements setting to 100. This is probably low, but the site doesnt get that much traffic and it's below the OPEN_CURSORS setting.

If any of you smart types know why we experienced this problem with and ODBC connection, please comment!

---

CFDOCUMENT top margin cuts into the first line

---

I posted this a while ago on StackOverflow.com but considering I answered it myself and my blog needs content, I' posting it here too.

We produce a lot of PDF output for our customers. On some of the pages, the top line was getting cut off often times making it unreadable. There's an example of this here. I know there was a bug in 7, but that was said to be fixed in 8 and we were using 8,0,1,195765.

After some testing I found that it only happens within table cells. The problem is, we're outputting tabular data. I tried using divs to creat the table using CSS

display:table-cell; display:table-row;

But that didn't work either.

A little more poking around and I found that wrapping <div> around the data inside the <td> tags solved the problem.

<td><div>#dateFormat(now(), "mm/dd/yyyy")#</div></td>

The one time this didn't work was because of nested table tags (which shouldn't have happened anyway). After fixing the nested tables it worked like a charm.

---

No empty strings in Oracle

---

This was pretty easy to find, but considering how unexpected it is, I feel it's worth a mention.

There are no empty strings in Oracle's SQL or PL/SQL.

Consider -

myVar VARCHAR2(10) := '';

The following will never execute.

IF myVar = '' THEN
-- do something
END IF;

myVar must be compared to NULL for the block to work.

IF myVar IS NULL
-- do something
END IF;

Likewise

SELECT someField
FROM   someTable
WHERE  someField = '';

Will always return 0 rows.

---