Headsplode

Life without Flash, as bad as you thought?

2011-01-06T13:02:00.001-05:00

As a ColdFusion developer and aspiring Flash/Flex developer I was pretty upset with Apple's decision to exclude Flash support from the iPad. I promptly went out and bought the first Android 2.2 phone I could in protest! I don't regret that, I love my Droid X. However, a couple months back I was gifted an iPad. My wife and I both got one. I don't care how bias you are, you can't argue with a gift.

As much as I want to dislike this stupid thing, I just can't. In fact, I love my iPad. It hasn't left my side since I got it. Aside from Flash, I haven't found anything I can't do with it, and iOS 4.2 cleared up some of the small annoyances I had with it prior to the update, primarily multitasking.

My biggest surprise is that the lack of Flash isn't as earth shattering as I thought it would be. I think I can count on one hand the number of times either my wife or I have been inconvenienced by it. Granted, it was a pretty big inconvenience. But the number of them have been small.

This is just my experience. I'm sure there are other use cases out there, so I'm curious to know how my fellow Adobe community iPad owners feel about the lack of Flash? Is it as bad as you feared? Worse? A few inconveniences? Or maybe you don't own one strictly because of it? Let me know.

Does a good PDF reader for Android exist?

2010-11-10T06:12:00.001-05:00

I want to get an Android tablet. I've been looking at the Notion Ink Adam for months now. I think it looks like the strongest Android tab to hit the market in the near future.

Last month I was given an iPad as a gift. Despite it's lack of Flash, multi tasking (which I hear is being fixed this week?), and all the other goodness found in Android, the iPad does most of what I want, very well thanks to the apps.

One of the things I do is read a lot of PDF e-books. Mostly stuff like Adobe documentation or text books with a pdf version on the companion CD. The iPad does this well, even the free iBooks app which is what I use most makes reading PDFs a treat.

On my Droid X I've tried a few readers, including Adobe's official reader, and from my experience... they all kind of stink.

Adobe's official reader is pretty good for small documents. It's quick and smooth but doesn't remember where I left or allow me to even set a bookmark which is a problem with 600 page e-books.

Beam reader trial just seems clumsy. It would rather bounce at the bottom of the page than continue to the next one. In text view you can't even get to the next page by flicking. There seems to be some sort of glitch in PDF view where the pages that were loaded into memory already start to look like TV static. It does seem to remember what page you exited on though. The $10 price tag seems high to me as well considering the bugs.

The quick office reader seems pretty OK. The bookmarks let you easily jump to chapters and you can manually input the page but it doesn't remember what page you exited on or allow you to create a bookmark.

There are some other pay options but I can't see paying for an app that may not work. Don't get me wrong, I don't mind buying apps, but I do mind buying apps that don't work. At some point I'd like to start developing for the Android so this would be a good project but until then, anyone have any input? Loading the last page read is a huge thing for me.

Apple and what used to be the best e-music service ever.

2010-09-01T14:31:00.001-04:00

So, I just finished watching Apple's music event. They talked about new iOS updates, iPod updates, iTunes 10, and iTV.

I tuned out most of it, except for the iTunes part, hoping to hear how Apple took advantage of Lala's streaming technology. I have yet to find a music service that offers what Lala had. Web songs for $.10, MP3 for $.89 (often less), social networking, discovery, full album previews, enormus selection, it was such a great service.

Sadly, it looks like the only thing they've taken from Lala so far is the social networking. iTunes 10 has a "Ping" feature that is basically facebook for music inside iTunes (not the web). Follow, be followed, share, discover, etc.

This doesn't mean iTunes won't eventually be a streaming service and offer what Lala had, but unil then there's a huge void in internet music services. Too bad google didn't bid for it more aggressively.

Don't be so quick to blame ColdFusion

2010-07-09T13:21:00.000-04:00

Not long ago my new boss and I were having a conversation and it came up that we might not be using ColdFusion much longer, but he’d get me trained on whatever I needed when they changed technologies.

Back story: The ColdFusion app that I’m maintaining now is an add-on to an asp.net application. It turns out the reason this add-on isn’t in .net is because the person they had available didn’t know or want to learn .net (I don’t blame him personally). On the home page of the .net app lives an iframe with a news scroller. The news is stored in a table and presented in a .cfm page. I’m sorry but I don’t see any reason why a single query and a few records can’t be displayed in .net, in a .net application. His aversion of .net was that strong.

When I asked why we were moving away from CF he said “Well, when we started this CF was the hottest thing, then not so much, now it is again, so who knows. CF also can’t handle the requirements. The system runs like shit!”

When I asked who said CF wasn’t the greatest thing anymore he said “the same guy that said it was great, the first developer.” I was pretty shocked. The same guy that didn’t want to learn enough .net to make a single page is now blaming CF for his lack of ability! Thanks to him, CF has a bad rap with my management and our customers!

The system did run like shit, fortunately for me one of my side projects is very similar to my day time job, making “charts and graphs 'n shit” as my boss puts it. I showed what I had done and how fast it works. I suggested it might not be CF, but the previous two developers. I asked him to give me the opportunity to fix the system before passing judgment on CF.

This week I put some proper error trapping in the system which logs the errors and a few other modifications. I saw a couple pages people were hitting often but were timing out. I made a couple changes to the query in question, removed the two queries nested inside the cfloop, removed # signs where they weren’t needed, removed some conditionals, replaced the several font tags with CSS, et cetera. The page loads in 1.3 seconds now. Using CSS also reduced the size of the .xls and html files by almost half (the xls files are html output with cfheader, I need to fix that too) so the page also downloads faster when 12,000 rows are returned (no, there’s no paging, customer requirement) from about 12 megs to 7.

Employers/managers: If your ColdFusion apps aren't running well, don't be so quick to pass judgement on ColdFusion. Look at your hardware, SQL, bandwidth and even your developers. If you're not a technical person, ask someone OUTSIDE of your team or higher a consultant to look at your situation.

I think I’ve saved ColdFusion’s reputation within my organization with these simple changes. My next hurdle is to save CF from our customers. Hopefully the proper error trapping and gradual fixing of the system will do this. Wish me luck!

MySQL Union and query optimization

2010-07-06T19:57:00.001-04:00

UNION can be a pretty nice tool. It’s a good way to query several tables with the same data with a single call to the database server. The most common example includes adding results from archive table to your recent data. MYSQL ships with two flavors of Union; DISTINCT and ALL (assume this is the same with Oracle and MSSQL but I don’t use them enough to know).

Basic syntax:
(SELECT fields FROM table WHERE conditions)
UNION [DISTINCT | ALL]
(SELECT fields FROM table2 WHERE conditions)
[WHERE]
ORDER BY
[LIMIT]

UNION defaults to UNION DISTINCT implicitly. The difference between DISTINCT and ALL is pretty self explanatory; DISTINCT prevents duplicates just like in a normal SELECT statement. The problem with DISTINCT is that it’s very VERY costly. Both DISTINCT and ALL build a temp table to store the results; however, DISTINCT must index the results to avoid duplicates. In addition to that, from what I’ve read, applying ORDER BY and LIMIT to the entire UNION will cause MYSQL to ignore any indexes as well. I have the need to UNION up to eight tables. In my test I was working with a query I know that was giving me problems; it has six tables.

Table 1: ~25,000 records
Table 2: ~8,000 records
Table 3: ~300,000 records
Table 4: ~200,000 records
Table 5: ~820,000 records
Table 6: ~125,000 records

My where clause was pretty simple, just looking for all the records added by a single user. UNION DISTINCT caused my ColdFusion page to time out after 60 seconds. I could increase the timeout to get an accurate picture of how long it actually takes but I don’t feel like breaking anything right now (that’s what my 9-5 job is for). At first I was afraid it was because MYSQL ignores the index when ordering and limiting unions. My first thought was I needed to have to run each query individually then do a QofQ to re-order them.

In my particular situation I'm not going to have any duplicates because the data in each query is already unique to the other tables. In this case I can safely use UNION ALL without getting any duplicates with the added benefit of the speed UNION ALL gives me over UNION DISTINCT. How much faster is it? Considerably. My query went from timing out at 60 seconds to completing in 34 seconds. That’s pretty sweet, but not great.

My needs require me to limit the end result to 500 records. That’s fine for my end result, but what about each SELECT within the union? Believe it or not 99% of all the records in these six tables match my WHERE condition. That means my unions create a temp table of ~1,500,000 records of which I only need 500. No wonder UNION DISTINCT timed out; that’s a lot of indexing. Building such a large table when I only need a very small subset of that is just silly. Fortunately I can limit the results from the individual selects. Limiting each select to 500 now gives me a temp table with only 3,000 records, much better than 1.5M and growing! Unfortunately, to be sure I get the results I expect, I also need to include the same ORDER BY clause to each select that I have at the end of the entire union. Even with the overhead of ordering 1.5M records, the query now finishes in 21 seconds, a savings of 13 more seconds.

Most of the time the query only runs using tables 1 and 2, which are pretty small so it runs very fast. If anyone has any other ideas on how to make the larger unions run better, please comment below!

Homemade Page Timing

2010-06-29T08:27:00.001-04:00

I started a new job a few months ago and inherited a mess. Don't believe me? Friday I found

<cfif isDefined("form.loginSubmit") and true>

in the login script.

I actually found

<cfsetting requestTimeOut = "240000" >

I mean seriously? 66 Hours???!

Anyway, the job consists of doing some pretty heavy data analysis of millions of records and hundreds of tables/views. Needless to say the queries don't run so hot. Some of the pages don't even run at all.

We're using CF9 enterprise; however, I don't have access to any of the logs or server monitor information so I had to make my own. This may also be helpful to anyone using CF Standard edition without server monitoring.

onRequest to the rescue! And in true CF fashion, it's pretty simple to do.

The initial requirement was to track what reports were being used and how often. That's easy enough. While I was doing that, I wondered if it would be possible to time how long the page takes to run. With several reports timing out it would be good to know these things. I had never tried doing any processing after including the targetPage, but I didn't see any reason why it wouldn't work.

<cfset startTimer = getTickCount()>
<cfinclude template = "#arguments.targetPage#">
<cfset timeSpent = getTickCount() - variables.startTimer>

After getting the timeSpent I saved the user, date, file name, and timeSpent to a new table. This afforded me the opportunity to get some analysis on our analysis. How ironic. I was able to create two quick views. The first is simply the data I saved highlighting the longer running reports by color. The other was the fastest, average, and slowest processing times and a hit count of the report.

Not to my surprise I found several pages that take 7+ minutes to process... I have my work cut out for me...

You're such a dork!

2010-06-28T12:43:00.001-04:00

...is what my wife said to me Saturday at Best Buy.
She might be right, but what else was I supposed to do? I felt it was my duty to educate the general public and entertain myself in a childish yet strangely satisfying way -- I kind of felt like Cereal Killer being generally mischievious (only not as cool).

The guy at the Apple table wasn’t looking so I made all the iPads look like:

and continued my shopping. I glanced over at the display on my way out and the screen was still up on all 4. It’s a shame I couldn’t set the homepage.

OnRequest required to access page variables scope in onError.

2010-06-15T22:59:00.000-04:00

I was playing around with onError the other day getting ready to upgrade some legacy code I just inherited. Because I was only concerned with onError I didn't bother to add onRequest during my testing. This is the first time I've never had onRequest in my application.cfc so I didn't realize there was such a dependence between onError and onRequest until then. Ben Nadel blogged about the same thing quite a while ago when he was missing the root cause structure after deleting the onRequest method.

I hadn't gotten to dumping the actual error contents yet, I was more concerned with the variables scope... which was void of the data I was expecting it to contain. I knew right away what the problem was and it makes total sense, but having never experienced it before, gave me one of those "no shit, Sherlock" moments.

The problem is that without <cfinclude template = "#arguments.targetPage#"> the page's variable scope is OUTSIDE of application.cfc so dumping #variables# only gives you the variables you set in your application.cfc. The cfinclude effectively brings the page processing into the cfc giving onError access to those variables.

If you're one of those people still using cferror, having direct access to the variable scope is worth the change by itself.

ColdFusion likes lazy programmers

2010-06-13T03:04:00.031-04:00

I don’t usually like to alias my code. Even when I write SQL I like to use the full field name, schema.table.column. I only use aliases when I’m in a sub query and need to reference the main query; then it’s required. I do the same thing in ColdFusion, I don't usually assign deep variables to a reference (no I don't really think it's lazy, just my preference).

Recently I was working with xml documents uploaded by users. For some reason my looping through my collection was timing out. My test document has 12,804 child elements from the root. At first I thought it was the processing I was doing within the loop. I kept removing sections of code trying to find out where the problem was. Finally I was left with a single cfoutput of a single element and my code was still timing out.

I’m not sure why I decided to try, because it isn't something I would normally do, but I assigned the parsedXML to a reference. To my surprise this solved my timeout problem. It solved it so well... my head exploded.

Here is a simple test to show the contrast.
The first loop uses the full variable syntax:
variables.parsedXML.XMLrootElement.XMLChildElement[variables.i].id.xmlText
The second loop uses the reference:
variables.XMLByRef[variables.i].id.xmlText

<cfsetting requesttimeout="960" /> 
<cfset variables.parsedXMLLen = arrayLen(variables.parsedXML.XMLrootElement.XMLChildElement) > 
<cfset variables.startXMLLongLoop = getTickCount() > 

#variables.parsedXML.XMLrootElement.XMLChildElement[variables.i].id.xmlText#


<cfset variables.stopXMLLongLoop = getTickCount() >

<cfset variables.XMLByRef = variables.parsedXML.XMLrootElement.XMLChildElement > 
<cfset variables.startXMLShortLoop = getTickCount() >

#variables.XMLByRef[variables.i].id.xmlText#

<cfset variables.stopXMLShortLoop = getTickCount() >

Fully qualified: #variables.stopXMLLongLoop-variables.startXMLLongLoop# miliseconds to loop #variables.parsedXMLLen# times
Referenced: #variables.stopXMLShortLoop-variables.startXMLShortLoop# miliseconds to loop #variables.parsedXMLLen# times

The result:
Fully qualified: 537765 miliseconds to loop 12804 times
Referenced: 250 miliseconds to loop 12804 times

The first loop using #variables.parsedXML.XMLrootElement.XMLChildElement[variables.i].id.xmlText# took almost 9 minutes to run!!
The second loop using #variables.XMLByRef[variables.i].id.xmlText# only took a quarter second??!! WTF just happened here? Anyone know why using a reference variable is faster than using the fully qualified variable?

Thanks!

ColdFusion and ORA-01000: Too many open cursors

2010-06-09T20:35:00.002-04:00

Today one of our sites started throwing an "ORA-01000: Too many open cursors" error. I have exactly 0 control over our server environment nor do I have access to the ColdFusion administrator. I have even less influence on our Oracle environment. We noticed the problem just before our daily scrum meeting so if nothing else, it got me out of that. Of course the first thing our sys admin did was contact our Oracle DBAs to see if they were having any problems (let the finger pointing begin, well actually I started it). They verified our data source connections and sent me a copy of the log files, which I already know weren't going to help much.

In my research, one of the Google results I came across made me chuckle. It was a Macromedia (now Archived Adobe) TechNote blaming the problem on Oracle. It said to change the OPEN_CURSORS and CLOSE_CACHED_OPEN_CURSORS settings.

It would appear that the Max Pooled Statements setting was increased in CF9, which we just upgraded too. If ColdFusion tries to keep more cursors open than Oracle will allow this error will occur. ColdFusion does this with queries using cfqueryParam and cfstoredproc to improve performance.

From Livedocs:

Max Pooled Statements enables reuse of prepared statements (that is, stored procedures and queries that use the cfqueryparam tag). Although you tune this setting based on your application, start by setting it to the sum of the following:
Unique cfquery tags that use the cfqueryparam tag
Unique cfstoredproc tags

Don't mistake Max Pooled Statements for the Maximum number of cached Queries under Server Settings > Caching, they're not the same. Max Pooled Statements can be found in the advanced settings of the data source (but I think only for SQL Server and the native Oracle driver).

The problem is, for some reason our data source was setup to connect using ODBC rather than the native Oracle drivers that come with ColdFusion Enterprise. There IS NO Max Pooled Statements setting for ODBC connections under advanced settings. Does that mean there's a similar setting in the background somewhere for miscellaneous databases? Was this setting increased in CF9? I don't know. I looked but can't find anything.

We finally got the problem solved by recreating the data source using the Oracle drivers and lowering the Max Pooled Statements setting to 100. This is probably low, but the site doesnt get that much traffic and it's below the OPEN_CURSORS setting.

If any of you smart types know why we experienced this problem with and ODBC connection, please comment!

CFDOCUMENT top margin cuts into the first line

2010-06-08T08:12:00.004-04:00

I posted this a while ago on StackOverflow.com but considering I answered it myself and my blog needs content, I' posting it here too.

We produce a lot of PDF output for our customers. On some of the pages, the top line was getting cut off often times making it unreadable. There's an example of this here. I know there was a bug in 7, but that was said to be fixed in 8 and we were using 8,0,1,195765.

After some testing I found that it only happens within table cells. The problem is, we're outputting tabular data. I tried using divs to creat the table using CSS

display:table-cell; display:table-row;

But that didn't work either.

A little more poking around and I found that wrapping <div> around the data inside the <td> tags solved the problem.

<td><div>#dateFormat(now(), "mm/dd/yyyy")#</div></td>

The one time this didn't work was because of nested table tags (which shouldn't have happened anyway). After fixing the nested tables it worked like a charm.

No empty strings in Oracle

2010-06-07T21:21:00.009-04:00

This was pretty easy to find, but considering how unexpected it is, I feel it's worth a mention.

There are no empty strings in Oracle's SQL or PL/SQL.

Consider -

myVar VARCHAR2(10) := '';

The following will never execute.

IF myVar = '' THEN
-- do something
END IF;

myVar must be compared to NULL for the block to work.

IF myVar IS NULL
-- do something
END IF;

Likewise

SELECT someField
FROM   someTable
WHERE  someField = '';

Will always return 0 rows.