Adding "Ajax Throbbers" to Zone updates

A common desire in Tapestry is for Zone updates to automatically include a throbber (or "spinner") displayed while the Ajax update is in process. This is, unfortunately, a space where the built-in Tapestry 5.3 Zone functionality is a bit lacking. Fortunately, it's not too hard to hard it in after the fact.

This solution involves a JavaScript library, two CSS stylesheet files (one is IE specific), plus the "throbber" image. Typically, you'll bind all of these things together in your application's Layout component.

First, the JavaScript. We need to intercept links and forms that update a Zone. When such a request starts, we add a <div> to the top of the Zone's client-side element. When the update from the server arrives, the entire content of the Zone's element will be replaced (so we don't have to worry about clearing the <div> explicitly).

When a form is submitted with Ajax, to update a Zone, Tapestry fires a client-side event on the Form; the Tapestry.FORM_PROCESS_SUBMIT_EVENT constant provides the event name. The primary handler for this event is the code that actually performs the XmlHTTPRequest and sets up a handlers for the response; the above code adds a second handler that adds the Ajax overlay.

Likewise, when a link is used to update a Zone, there's a second client-side event; again, the primary handler for the event does the actual Ajax work, but the same logic allows the Zone to be decorated with the overlay.

The overlay consists of a <div> that will visually mark the entire zone's content and consume any mouse clicks during the Ajax update. The CSS associated with the zone-ajax-overlay CSS class sets up a translucent background color and the spinning Ajax throbber.

Next up is the CSS:

This little bit of CSS is doing quite a bit. Firstly, if the Ajax request is very quick, then there will be an annoying flicker; to combat this, we've set up a simple CSS animation to delay the animation momentarily, long enough that fast requests will just see the new content pop into place. There's probably a bit of room here to tweak the exact timing.

Alas, in the current world, we need to do a bit of work to support both Firefox (the -moz prefix) and WebKit (Safari, Chrome, the -webkit prefix). This is really calling out for a SASSy solution.

You'll also see an animated image for the throbber. I used ajaxload.info to create one.

But what about Internet Explorer? It doesn't understand the animation logic, and it does CSS opacity differently from the others. Fortunately, we can segregate those differences in a separate CSS file.

Lastly, we put all this together inside the application's Layout component:

The @Import annotation does the easy imports of the main CSS and JavaScript.
Tapestry 5.3 supports IE conditional stylesheets ... but this requires just a bit of code as the @Import annotation doesn't support adding a condition, as this is a fairly rare requirement.

Instead, the IE-specific CSS is injected into the page as an Asset object; this can be combined with StylesheetOptions to form a StylesheetLink, which can be imported into the page.

With this in place, every page will include both CSS stylesheets (one as an IE-only conditional comment) and the necessary client-side logic ... and every Zone update will get this uniform treatment.

There's some limitations here; in Tapestry it's possible for the server-side to push updates into multiple Zones. The client-side doesn't even know that's happening until it gets the reply, so there's no universal way to add overlays to multiple zones when the request is initiated.

Secondly, in rare cases, a Zone update may only update other Zones, and leave the initiating Zone's content unchanged. In that case, you may find that the Zone's throbber is still in place after the response is handled! I'll leave it as an exercise to the reader on how to deal with that.

Dissecting a Tapestry Operation Trace

I'm helping out a client who is having a problem using Spock and Tapestry 5.3 together. The Spock/Tapestry integration was created for Tapestry 5.2, and some subtle change in the Tapestry 5.3 IoC container has boned the integration, so running even a simple test results in an exception, with a very big stacktrace:

The part at the top (the numbered entries) are the operation trace: Tapestry tracks what its doing at all times (using a stack stored inside a ThreadLocale) just so it can report this information in case there's an error. That's part of Tapestry's commitment to useful feedback. The operation tracing was enhanced a bit in Tapestry 5.3 to be a bit more verbose and all-reaching.

The operation trace is providing a wealth of information about how Tapestry got to the point where an exception was thrown. This is much more useful than just the huge stack trace (about 400 frames!) since Tapestry, by design, tends to call through the very same methods repeatedly; stack traces are less useful when what counts are the parameters to the methods, rather than the methods themselves.

It takes a while to figure out, but the key operations are:

That operation corresponds to invoking this method:

Translated to English, this code says:

When starting up the Registry (the odd name for the Tapestry IoC container), execute this block of code, that checks to see if early startup of Hibernate is desired and, if so, forces the initialization of Hibernate by invoking the getConfiguration() method (otherwise, the initialization would happen lazily the first time a request needed to use the Hibernate Session).

The @Symbol annotation means that the parameter's value is derived from a Tapestry configuration symbol, which is a flexible, late-binding way to configure services as they are instantiated. In other words, because of the use of a symbol, rather than a constant, the actual value passed in can't be determined until runtime ... which is a good thing; it means a developer can configure the symbol's value locally, but a different default value is in effect for the production application. Sometimes you want early startup, sometimes you don't.

In order to resolve the value of a symbol, Tapestry must instantiate the SymbolSource service; it has its own configuration that depends on other services, including ApplicationDefaults, FactoryDefaults, as well as a few other simple objects that implement the SymbolProvider interface, but are not services.

There's also a hidden dependency here: Starting in Tapestry 5.3, Tapestry will attempt to type-coerce contributed values (in this case, symbol values contributed to ApplicationDefaults or FactoryDefaults) from their actual type, to the expected type. That shows up in operation 14:

Those integers and booleans need to be converted to Strings; Tapestry 5.3 invokes the full machinery of the TypeCoercer service to do this coersion, seen as operation 15.

At operations 21 - 23, Tapestry sees the ThreadLocale service (which stores the active Locale used during processing of the request; something that can vary on a request-by-request basis). The ThreadLocale service uses a special service lifecycle that enforces that the instance is stored as a per-thread singleton, not a per-Registry singleton, and will be discarded at the end of each request.

The ServiceLifecycleSource service is the source for these ServiceLifecycle objects.

At operation 28 - 31, the Spock/Tapestry integration code is getting involved. It adds a special service lifecycle just for values that are part of a Spock specification ... and we're finally reaching the problem point!

The Spock/Tapestry integration is using the addInstance() method, which instantiates a class with dependencies; this is operation 30. This is the problem point, but it's not obvious why its causing an eventual exception.

Because of the use of addInstance(), Tapestry must locate and inject the dependencies of the PerIterationServiceLifecycle class, including the IPerIterationManager service (operation 31).

In Tapestry, there is a mechanism to replace services with overrides; this is the ServiceOverride service and its configuration. It's super handy for extending Tapestry in completely unexpected ways.

That brings us to some code, new in Tapestry 5.3, at operation 38:

And that brings us to the actual cause. Notice the @Symbol annotation ... remember way back to operation 7, that required the TypeCoercer (operation 15) ... well, we're not done with that yet, but this production override code has a @Symbol annotation that requires the TypeCoercer ... which is still in the middle of being instantiated.

Yes, this takes a lot of detective work ... this is something of an Achilles' Heel of Tapestry's IoC container; since much of the functionality of the container is defined in terms of other functionality of the container, you can get into these hidden dependency cycles when tweaking some of the more fundamental aspects of Tapestry, such as TypeCoercer contributions, or adding new service lifecycles. This is unfortunate, since so much else in Tapestry's web framework and IoC container Just Works^TM.

In terms of fixing this ... turns out the Spock/Tapestry integration has some other dependencies on 5.2, making use of internal classes and constructors that no longer exist in 5.3. I'll be forking their code shortly to produce a 5.3 compatible version.

However, my take-away here is: the system works, the emphasis on feedback, and the generation of useful operation traces, makes this detective work even possible. The alternative would have taken far, far longer ... using the debugger to try and work backwards to what Tapestry was trying to do. It's so much better to have Tapestry simply tell you what you need to know!

Mac Tips: Preventing Aperture from Launching

Since I tend to connect and disconnect my Android phone to my Mac pretty often, I got frustrated that it kept launching Aperture every time (it used to do the same with iPhoto, before I switched).

In any case, the solution for this is easy enough; with the phone connected, launch the ImageCapture.app: select the phone, and choose "No application" from the drop down list in the bottom left corner:

This is under Lion; your mileage may vary under earlier versions. Hope it helps!

Gradle Training in Portland

It's no mystery that I'm a big Gradle fan ... it's by far the best build tool I've even seen. Build scripts are concise and readable, and Gradle the easiest build tool to configure and extend, as well.

I fought an uphill battle with the other Tapestry developers to replace Maven with Gradle. Yes, we had an existing Maven build and it worked (most of the time, until it didn't). Now past the switchover, we're really reaping the benefits: faster builds, fewer headaches, and it's much easier to add new sub-modules. Despite the achievements I've made with Gradle, there's a lot about this powerful tool I still need to learn. So I was surprised and pleased when Tim Berglund contacted me to help find a venue in Portland, Oregon for Gradle training.

They're now advertising that the three-day training is coming January 17-19, 2012; details here.

I've seen Tim speak at a number of No Fluff Just Stuff and Uberconf sessions; he's very good. If you are frustrated using Ant or Maven, you need to learn Gradle, and I can't see how getting an intensive brain dump won't do anything but save you time and money.

Some Tapestry Stories

A ways back, I published a call for anyone interested in free Tapestry 5 Laptop stickers (that call is still open!). You get the stickers, I get a story. Here's a few highlights, in no particular order:

Robert B., USA

The SC Medicaid Web Portal enables doctor's offices and hospitals, using the Web, to enter and submit claims for patients enrolled in Medicaid in South Carolina.

Steve E., Singapore

We're producing an on-line matching system for Non Deliverable Forwards (NDFs) in the currency finance market for a brokerage firm. (Or I guess in layman's terms, a gambling system!)

Kejo S., Switzerland

We are working now since almost 3 years with T5 and we created an application platform for ABB with it and really a lot of other projects. I think we wrote already far more than 1 million lines around your framework! Often your framework is fun and sometimes pain, but it's always amazing how lean and clean your source is! Overall I think it is the best frontend framework in the java ecosystem right now! Great Work!

Ivan K., Belarus

We just started new project on t5 - online collectible card game.

Michael L., Germany

We started a Tapestry5 project to build our extranet application, that links into the ERP to provide realtime information to stake holdersaand supporting internal workflows. We're just at the beginning and implementing more and more stuff. However, we looked at different Web-Frameworks but Tapestry5 simply rocked!

Greg P., Australia

I'm using Tapestry to create liftyourgame.com. A site that allows people to achieve their goals.

Szymon B., Poland

We use Tapestry 5 in our economic information service neurobiz.pl. It gives users access to the information about businesses operating in Poland and registered in National Court Registry.

Dominik Hurnaus, Austria

Working on a large CRM system for an automotive customer.

James S., USA

My team and I are working on a web-based interface for a fuzzy lookup and matching engine we've developed. I've also started messing around Tapestry for a few of my personal projects. I started using T5 a couple months ago, and so far I'm loving it.

Nenad N., Serbia

I am working with 5 other developers on mobile portals developed with Tapestry for multiple clients.

Dragan S., Macedonia

I was a GSOC developer and now I'm trying to do new cool stuff with Tapestry like websocket integration with node.js and rabbitmq.

Volker B., Austria

Our project is a dealer management system which supports dealers and workshops of the VW Group's brands and the Porsche sports car brand in all sorts of operational processes in a modern and flexible way ... in our company I think there are about 80-100 people that are using Tapestry.

Daniel J., Canada

Assessment dashboards for schools in southwest SK, Canada

William O., USA

We are working on a number of cool Facebook apps using Tapestry. One's called My Social Rankings ( mysocialrankings.com ), and the other is called Blingville (blingville.com).

Peter P., Slovakia

We are developing web applications for broker companies using Tapestry 5, and its great to develop with Tapestry.

Pablo N., Argentina

We are using Tapestry for http://www.squidjob.com (migrating out of GWT). The site is THE place for finding service providers for anything.

Joost S., the Netherlands

Yanomo is time tracking, project management and invoicing software for the rest of us. Use it and "You know more" :)

Alexander G., Belarus

We have been using Tapestry for about 6 years in our projects. Our current project is web administration console for RadiumOne Display (www.radiumone.com) platform. We are very happy with our stack consisting from Tapestry5+Spring+Hibernate+jQuery.

As usual, I see a lot more Tapestry adoption outside the US; I wonder if its about programming culture ... or about Tapestry localization support? I tend to see the Europeans developers as having more freedom to work with less mainstream technologies ... but when I ask them about this, they always seem to think that it's the US developers who have that freedom. I guess the grass is always greener.

Tapestry 5.3 Maven Archetype

After struggling most of yesterday day, I finally have the Maven archetype working.

Use the command:


mvn archetype:generate -DarchetypeCatalog=http://tapestry.apache.org/


This points Maven at the Tapestry-specific Archetype Catalog, you want the first option ("Tapestry 5 Quickstart Project"), then choose the fourth option ("5.3-beta-20").

I need to fix something broken in the CSS of the archetype; otherwise it's working nicely, and demonstrates a bunch of new features in Tapestry 5.3; it also makes use of up-to date features, such as using the @Import annotation (instead of the now removed @IncludeStylesheet annotation), and some Ajax features on the first page.

I may switch things around to make use of Twitter's Bootstrap CSS stylesheets.

Here's a full session of creating the application and getting it running; it took less than 30 seconds:

Things You Didn't Know About Tapestry 5.3

I missed the JavaOne Comparing Web Frameworks talk and was appalled at some out-of-date information in it ... though reviewing his slides, it looks like he talked about Tapestry 5 but showed out-of-date Tapestry 4 examples. With Tapestry 5.3 ready very soon now (and less than a year after 5.2) it seemed like a good time to share some cool things about Tapestry:

Tapestry Release Compatibility

Tapestry 5.3 will be available soon; and for the majority of users, the upgrade is simply a matter of changing the version number in their pom.xml or build.gradle. Release compatibility was certainly a major headache from Tapestry 3 to Tapestry 4; and there is admittedly no direct upgrade path from Tapestry 4 to Tapestry 5 ... but Tapestry 5 was created from the ground up to prevent the kind of pain experienced in prior Tapestry releases. To wit:

• Services and dependency injection allows a fine-grained, not monolithic, approach to evolving the framework
• Metaprogramming of components allows new behaviors to be gradually introduced, bypassing the fragile base class problem
• Tapestry 5 has always carefully separated internal interfaces (which is not guaranteed to be stable between releases) from stable public interfaces. Literally, in packages named .internal. so there's no guesswork about what's public and what's internal
  

Tapestry Makes It A Snap To Work With Hibernate and JPA

Tapestry has built-in modules for supporting both Hibernate and JPA. You get lots of stuff for free, including automatically configuring entities (just put them in the right package), easy transaction management, and smart encoding/decoding of entities ... Tapestry knows how to convert back and forth between entity instances and primary keys when building URLs.

Tapestry Works Great With Spring

Tapestry integrates very cleanly with Spring. You can inject Tapestry services into Spring beans, you inject Spring beans into Tapestry services and components.

Tapestry Plays Well With Others

Tapestry doesn't care if you have other servlets running in the same web application. You can fine-tune how it builds URLs, or even put Tapestry in a box so it doesn't collide with other servlets or filters. You can also easily share information with other applications.

Tapestry Likes HTML5

Starting in Tapestry 5.3, <!DOCTYPE html> works perfectly in Tapestry.

Tapestry Hot Deploys

Tapestry (since 2006) has had live class reloading; change a template or a Java file and Tapestry reloads it instantly. And since its integrated into the framework, Tapestry can be very efficient about loading and reloading resources. Since 5.2, Tapestry has also live reloaded (most) service implementations. So code away! Tapestry can keep up with you.

Tapestry Loves Ajax And JavaScript

Tapestry (currently) bundles Prototype and Scripaculous, but you can swap that out for jQuery quite easily. Tapestry has most common Ajax use-cases built in, and uses a uniform approach to rendering full pages, or individual snippets. Tapestry does a lot of other tricks, such as combining your individual JavaScript files into a single JavaScript stack (on the fly, at runtime). In addition, Tapestry has an extensible framework for organizing your JavaScript and initialization code (partly on the server-side, partly on the client-side).

Tapestry 5.3 adds vastly improved reporting of server-side exceptions, along with an easy way of presenting alerts to users.

Tapestry Is Polyglot

Tapestry doesn't care if your classes are written in Java, Scala or Groovy ... if it's bytecode, that's all that counts.

Tapestry Is Fast And Getting Faster

Tapestry has been getting faster and leaner with each release. 5.2 introduced page singletons (where a single page instance can be safely shared across many threads, even though it contains mutable fields) and 5.3 boosts the performance in a bunch of ways large and small. Tapestry 5.2 scored right at the top of this performance comparison, and Tapestry 5.3 is around 30% faster.

Tapestry Has The Best Feedback Of Any Framework, Period

Tapestry's approach to feedback goes far, far, far beyond any other framework or toolkit; it goes beyond the comprehensive exception report page and extends to small concerns throughout the framework:

• Tracking what the framework is doing, and why, at all times
• Including extra checks for common errors and building real messages that identify what went wrong and how to fix it
• Built-in pages to allow simple application monitoring

Tapestry Really Gets Localization

Localization support isn't an add-on; it's built-in from the ground up. Tapestry allows templates and other assets to be localized automatically: just follow the naming convention and Tapestry uses the correct file. Tapestry has localized messages for 14 languages and counting.

Tapestry Is Customizable

Tapestry's architecture, based on lots of individual services and dependency injection, means that almost any service or other logic in Tapestry can be overridden. Don't like how Tapestry builds URLs? Replace it seamlessly. Dont' like how Tapestry reports exceptions? Replace it! Tapestry is designed specifically so that you can augment or replace any behavior in the framework.

Tapestry is a Meta-Programming Monster

And I mean that in a good way; Tapestry has powerful support built-in for meta-programming at the services layer and at the component layer. Tapestry lets you get in and modify method invocations and field access, without getting your hands dirty with the ugly bytecode details. All the cool things Tapestry does with naming conventions and annotations is wide open for application-specific things. Meta-programming provides a critical alternate avenue of code reuse.

Tapestry IoC Works Great On Its Own

Tapestry's IoC library works great on its own, separate from the web framework itself. That includes live class reloading, meta-programming capabilities ... even a simple job scheduler.

... that's enough for now. The point is that Tapestry has a lot going on ... to paraphrase Philip Greenspun:

Any sufficiently complicated servlet or JSP web application contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Tapestry.

... and half is being very generous!

Tapestry: Feedback!

I often say that the three cornerstones of Tapestry are Simplicity, Consistency, Efficiency, and Feedback. Although all four of these concepts work in concert with each other, it's Feedback (keeping the developer informed when things go wrong) that is one of the most distinguishing features of Tapestry, and that's only gotten better in Tapestry 5.3.

Exception Reporting

First off, there's Tapestry's default exception report page. When an exception is thrown during a request, most often a coding error in a Tapestry page or template, Tapestry moves heaven and earth to report the exception properly. For example, in Tapestry it is not allowed for a component sub-class to define a parameter with the same name as a parameter from a base class, as this creates an ambiguity. When this situation occurs, an exception is thrown from deep in the bowels of Tapestry:

That's a start, but it's not great feedback; you'll be doing a lot of work to figure out what was going on in Tapestry leading up to the exception, and from there, figuring out how to fix it; there's lots and lots of noise in the repeated stack traces (caused by nested exceptions). However, you can see a glimmer of hope in those first few lines, the ones that start Registry [ 1], Registry [ 2], ...

Tapestry goes to a lot of trouble to track what is going on during the handling of a request; it keeps a stack of operations which describe what Tapestry is doing at any particular time. There's still a lot of internal details, but the gist of it is that Tapestry needed to create an instance of the ParameterConflictDemo page, and hit an error while doing something with the ParameterSubClass component (sorry for the ugly names, I'm using examples from Tapestry's internal test suite).

However, parsing apart the console output is NOT what a Tapestry developer does; instead they'll get all those details, and more, from the Tapestry exception report page:

Ah, much better. We're seeing the essential details from the stack of exceptions; we're seeing the associated template snippet that defines the parameter with the exception, we're seeing that operations stack neatly formatted. We see that stack of operations here as well, formatted for readability. Scrolling down, we see the stack trace of the deepest exception, formatted:

The frame in bold blue? That's a frame in the application's package, rather than code in inside Tapestry. That highlighting is very useful for letting the developer quickly figure out if the cause of the exception is a minor problem inside their code, or something more involved that shows up inside the Tapestry framework code.

Scrolling further down, we start seeing even more relevant information: all the details of the incoming request:

... well, you get the idea. Where a framework that takes feedback less seriously might give you a simple stack trace and leave the process of determining the underlying cause entirely up to you (after all, you have a debugger, right?) Tapestry fully embraces the importance of feedback: giving you all the information you need as soon as you need it (and yes, you don't have to show all that to your end users). More than that, there's attention to detail throughout Tapestry to provide real exception messages. For example, if you provide a component type name that doesn't match some component, Tapestry responds with a detailed message, including a list of all the known component types:

Again, Tapestry doesn't want you to have to put on your detective's hat to figure out what's wrong and how to fix it. It's providing all the details you need right when you need them.

Live Application Introspection

What if your problems are more subtle? How do you track down other issues, like performance problems or memory utilization? Well, Tapestry provides some introspection to address those questions as well. Tapestry 5.3 adds the Page Catalog, a special page for providing information about what pages have been loaded into memory, and a few details about how big they are, and how long they took to assemble:

Likewise, the Service Status page gives you feedback about the services defined inside the Tapestry Inversion of Control container, helping you determine what services exist, and in what state:

Deep Details

And what if you are tracking down something even more subtle? Well, by enabling some logging, Tapestry will output that operations trace as each operation starts and finishes. It's an avalance of information, much of it about instantiating services ... below is a log of just what happens when you first startup a Tapestry application, before it even processes it's first request:

Because Tapestry operates lazily, instantiating pages and services only as needed, even more happens when the first page request arrives:

Conclusion

In any case, the point of all this is that Tapestry provides you with the key tool, information, at all stages of development. This is central to Tapestry as a tool to be used: a framework that gets in the way, that makes any aspect of development harder or slower than it should be, is a framework that should not be used ... and I feel quite strongly that Tapestry is a framework meant to be used!

Changes to Cascade, and a cautionary tale about defrecord

Since I've been talking more about Clojure lately, I've spent a little more time working on Cascade. I've been stripping out a lot of functionality, so that Cascade will no work with Ring and Compojure, rather than being in competition. Its Clojure, after all, ... there's less of a reason to build a full framework since its so easy to simply assemble your functionality from proper libraries.

It's also been a chance to update some of my code with more modern constructs. For example, the earlier version of Cascade used a (defstruct) for the DOM nodes; the new code uses (defrecord).

Along the way I discovered something interesting about defrecord. Consider this code:

Technically, this is just an optimized way to define a Clojure Map. If I have an instance, I can (:text node) to get the text out of the map.

However, (defrecord) does one other thing that is barely mentioned in the documentation (and not referenced, that I can tell, in Joy of Clojure). Notice the implementation of the stream function (part of the NodeStreaming protocol). It just says text; not (:text node). Inside a protocol method, the fields of the record are bound to local variables, making them easy to use ... another benefit.

I actually found this the hard way, when writing a more complicated example, for the Element DOM node:

Notice the use of clojure.core/name to convert a keyword to a string; originally this was (name (:name node)) and returned nil. This confused me quite a bit!

What ended up happening was that name was bound to the keyword from the record's name field. However, Clojure keywords can be used as functions,and was incidentally passed itself, which is to say (for an Element node representing a <p> element): (name (:name node)) --> (:p :p) --> nil.

So, (defrecord) giveth, but it also taketh away, at least, the first time. In other words, watch out for name collisions between the names of the record's fields, and the names of functions you want to reference from your protocol method implementations.

Back to Cascade; I don't have any metrics available about performance changes with the new code (using records and protocols), but I suspect its faster and more efficient.

A lot of the features that were in Cascade are gone and will come back soon. Ultimately, I'll have Cascade flavors of context and classpath assets from Tapestry, as well as mechanisms similar to Tapestry for adding JavaScript libraries and CSS stylesheets, along with a mechanism similar to Tapestry for organizing them into stacks.

Looking further forward, adding support for Enlive, both reading parsed XML templates in as DOM structure and allowing Enlive transformations onto the DOM structure, seems like a good direction.

When will all this happen? I'm not certain, but I hope that Cascade will become a "must-have" layer on top of Compojure, adding some of the industrial strength concepts from Tapestry into the fast-and-loose world of Clojure web applications.

Tapestry 5 Laptop Stickers

I'm the kind of guy who does a lot of self-expression on my laptop:

Hey, what's that at the bottom? A Tapestry 5 laptop sticker? Cool!

Want one? Drop me a line at hlship at gmail dot com with the subject "T5 STICKER". Provide me with a quick (one-line summary) of your project and a snail-mail address (outside the US is OK) and how many laptop stickers you need (keep it reasonable, as in, how many of your project developers will actually put one on their laptop).

First come, first serve, supplies are limited, offer may be rescinded at any time!

Update: I'm really enjoying seeing all the stories of people using Tapestry; if it's OK, I'd like to generate a list of people (just first name and last initial), country, and a one-line project summary, to post on this blog, or on tapestry.apache.org.

If you do NOT want to be publicized, please indicate that in your email.

How They See Us, How We See Them

Stumbled across this great graphic on GlobalNerdy Blog.

Sorry my blog layout gets in the way, try the direct link to the image.

Clojure: Towards the Essence of Programming

My talk from the What's Next Paris conference, Clojure: Towards The Essence of Programming is now online at InfoQ. This is the full talk ... video plus slides. See if you can spot the point where I almost pass out from jet lag!

Tapestry 5.3 Ajax Exception Reporting

I just put together this screencast about an exciting improvement to how Tapestry 5.3 presents server-side exceptions to the client.

Tapestry is certainly moving further and further into the rich client space; I think there's some compelling features of Tapestry that make splitting the application across the client web browser and the server quite attractive, including a uniform approach to rendering (both traditional page oriented requests, and Ajax partial page renders). In any case, the weak link in the chain used to be that with Ajax requests, server-side exceptions sent you scurrying to look at the console, and you would lose (along the way) a lot of the power of the Tapestry exception report page; now you get to have your cake and eat it too.

The real excitement will be coming in Tapestry 5.4, which will push much deeper into improved JavaScript and Ajax support, including a move to framework agnosticism (as in, switch over to jQuery seamlessly). Part of that support is already in 5.3, with more to come.

GroovyCasts

If you are a fan of Groovy, or even just curious what the fuss is about, there's a new resource for you: Groovy Casts. Groovy Casts is a blog by my good friend Merlyn Albery-Speyer containing short (five minute or less) screen casts about using Groovy, or about interesting and useful Groovy tools and libraries.

Some of the screen casts are done by Merlyn, others are simply collected by him from all over the web. Check it out!

Thanks Luke Daley!

Just wanted to reach out and thank Luke Daley, who just joined Gradleware, for giving me a big help with my Gradle build issues. We had a nice chat, did a little screen-sharing, and worked out how to get Tapestry to create the necessary PGP signature files for uploading. I can't imagine any ordinary company offering that level of support to an ordinary user ... or even a user with an expensive support contract. Kudos!

But I don't want to scare anyone away from Gradle ... I'm having to pencil pretty far outside the lines for a few specific Apache.org requirements, things that most people will never have to deal with. Even so, many of the problems I'm hitting will be correctly addressed in the future, possibly in the final 1.0 Gradle release. I'm now using Gradle in all my projects (for my clients and for my open source projects) without any significant problems ... and I encourage you to do so as well!

What's Next? Paris Clojure Slides

Here's my slides from the What's Next? conference in Paris. I found a couple of minor errors on review, and I'm expecting to reorganize it a bit before my next talk.

Clojure: Towards The Essence Of Programming (What's Next? Conference, May 2011)

View more presentations from Howard Lewis Ship

The Tragedy Of Checked Exceptions

If you ever get one of those interview questions along the lines of "What DON'T you like about Java?", I would hope that checked exceptions are at the top of your list. I think no other, ahem, feature, of Java has caused more code bloat, more problems, and less stability than checked exceptions. Java was the first main-stream language to include this concept (to my knowledge), the only (widely used) programming language that has it, and I strongly hope it will be the last programming language to include it.

Checked exceptions cause a lot of grief. They often pollute APIs: look at how the JDBC API plays "cover your ass" by making every single method throws JDBCException regardless of which methods do any work that could possibly fail. At best, checked exceptions make sense only when there is a clear and documented way to recover from the exception (such as waiting and retrying the failed operation). In many cases, a simple boolean, indicating success or failure, would be much better than an exception, and accomplish the same goals with far less cost.

Checked exception's also encourage another terrible pattern: exception swallowing. "What do I do with this here MyAPIIsBrokenException? Well Eclipse just inserts code to print out the stack trace, so that's good enough." Thus real errors get discarded, and code that should break during testing slips through the cracks, causing nasty runtime failures and ominous messages to the console.

Really, what can you do with an exception? Either handle it locally and immediately, or wrap it in another exception, usually RuntimeException, and rethrow it ... but that approach is only effective if some higher layer does a good job of reporting the entire stack of exceptions, the way Tapestry does. More often, the exception just percolates up to a top-level loop and spews out a few hundred lines of glop onto the console or log.

I think part of the proof that checked exceptions are simply unworkable is the way throws Exception is creeping into standard APIs, such as the ones specified in project Coin (I'm thinking of Autocloseable). And what is the semantic value of throws Exception? It's useless ... because you are either going to log that exception to the console or wrap it in a new RuntimeException and re-throw it. So the authors of Autoocloseable have simply shifted work onto your lap (you get to write the code to catch it and rethrow it) when if they simply omitted the throws clause, and documented that "close() methods may throw a runtime exception" you could get the exact same effect, but write much less code.

I've also seen that checked exceptions have been a factor in the delays for JDK 8 Lambdas, complicating that specification much further than it needed to be, and forcing new and odder syntax into the language to accompany it.

Meanwhile, post-Java JVM languages ... including Groovy, Fantom, and Clojure ... simply ignore checked exceptions; which is easy enough to do as they are almost entirely a fiction of the Java compiler in the first place. You can write try...catch blocks in these languages, but there's no pressing need to, and application stability ends up being higher than in traditional Java code.

It is unfortunate that of all the ideas that Gosling, Joy, and folks had at the dawn of the Java language, they deferred ones we've really been missing (such as reified types and lambdas) and included truly experimental features, such as checked exceptions. But that's just hind-sight and second-guessing. The real tragedy is that, unlike (for example) JavaScript, with Java you can't just use the good parts. Instead, Java snares you with an almost irrational compulsion to preserve the early mistakes in the language, forever.

Learn more about Tapestry power features @ Java Magic

New Tapestry user "tawus" has been working his way though the power features of Tapestry and Tapestry IoC on his blog: Java Magic. It's an interesting approach, he's covering one feature of Tapestry on each post, and showing what a (simplified) implementation of that feature would be. I'd rather he spent a little introduction time on why these features are useful and important, but it's still a very nice effort.

Extending JavaDoc

I don't think I've seen a piece of code more poorly designed for extension, and more in need of it, than JavaDoc. I'm in the process of removing Tapestry's Maven-based component report (as part of an overall move from Maven to Gradle). My goal is to merge what currently comes from the component report directly into the JavaDocs.

My first approach was to extend the built-in HtmlDoclet , the one that generates the every-day HTML report. Good luck with that ... it's like a field guide to anti-patterns for preventing extensibility. Here's an example:

public class HtmlDoclet extends AbstractDoclet {
    
    /**
     * The global configuration information for this run.
     */
    public ConfigurationImpl configuration = 
        (ConfigurationImpl) configuration();
    

    /**
     * Create the configuration instance.
     * Override this method to use a different
     * configuration.
     */
    public Configuration configuration() {
        return ConfigurationImpl.getInstance();
    }

    ...
}

public class ConfigurationImpl extends Configuration {

    /**
     * Constructor. Initialises resource for the
     * {@link com.sun.tools.doclets.MessageRetriever}.
     */
    private ConfigurationImpl() {
        standardmessage = new MessageRetriever(this,
            "com.sun.tools.doclets.formats.html.resources.standard");
    }
    
    public static ConfigurationImpl getInstance() {
        return instance;
    }

   ...
}

So, HtmlDoclet commits the cardinal sin of down-casting from the interface to the implementation class, and ConfigurationImpl is effectively final, as its only constructor is private. But you are encouraged to override the configuration anyway (recommended only if your goal is to throw a ClassCastException).

JavaDoc is old. The HtmlDoclet is just ... tired. Someone failed to tell these folks about XML and XSL, for example ... or about HTML and CSS, for that matter. JavaDoc is screaming out to be a tool that generates an XML representation of Java source content that can then be transformed into an HTML document tree via XSLT. I've seen an abandoned project along those lines. Perhaps in my spare time ... it would be a fun little side project to create that, and create a really world-class JavaDoc.

In any case ... I've been forced to impose the use of a @tapestrydoc tag on component classes that wish to be documented. Not the end of the world, but not backwards compatible either (though the Tapestry 5.2 Maven component report will continue to work with Tapestry 5.3, so that's not a deal-breaker).

An Example Of Why I Like Spock

Spock is really making writing tests fun, instead of a chore.

    @Unroll("toClass '#javaName' should be #expectedClass")
    def "toClass tests"() {
        expect:

        PlasticInternalUtils.toClass(getClass().classLoader, javaName) == expectedClass

        where:

        javaName | expectedClass
        "java.lang.String" | String.class
        "java.lang.Integer[]" | Integer[].class
        "java.lang.Long[][]" | Long[][].class
        "void" | void.class
        "int" | int.class
        "int[]" | int[].class
        "float[][]" | float[][].class
    }

This combines lots of things I've seen before in TestNG, but nicer; just more readable ... and the @Unroll annotation (which guides Spock on how to report the test execution) is really handy, especially when things go wrong. It's just a slick overall package.

Combining Gradle with Antlr3

I've been going through a relatively painless process of converting Tapestry from Maven to Gradle, and am thrilled with the results. My biggest stumbling point so far was Tapestry's use of Antlr3 for its property expression language.

The built-in support for Antlr only went as far as Antlr2. The Maven plugin I had been using understood Antlr3. After a bit of research and hacking, this is what I came up with as a solution for Tapestry:

description="Central module for Tapestry, containing all core services and components"

antlrSource = "src/main/antlr"
antlrOutput = "$buildDir/generated-sources/antlr"

configurations {
  antlr3
} 

sourceSets.main.java.srcDir antlrOutput

dependencies {
  compile project(':tapestry-ioc')
  compile project(':tapestry-json')
  
  provided project(":tapestry-test")
  provided "javax.servlet:servlet-api:$servletAPIVersion"

  compile "commons-codec:commons-codec:1.3"

  // Transitive will bring in the unwanted string template library as well
  compile "org.antlr:antlr-runtime:3.3", { transitive = false }

  // Antlr3 tool path used with the antlr3 task
  antlr3 "org.antlr:antlr:3.3"
}

// This may spin out as a plugin once we've got the details down pat

task generateGrammarSource {
  description = "Generates Java sources from Antlr3 grammars."
  inputs.dir file(antlrSource)
  outputs.dir file(antlrOutput)
} << {
  mkdir(antlrOutput)
  
  // Might have a problem here if the current directory has a space in its name
  
  def grammars = fileTree(antlrSource).include("**/*.g")
    
  ant.java(classname: 'org.antlr.Tool', fork: true, classpath: "${configurations.antlr3.asPath}") {
     arg(line: "-o ${antlrOutput}/org/apache/tapestry5/internal/antlr")
     arg(line: grammars.files.join(" "))
  }
}

compileJava.dependsOn generateGrammarSource

The essence here is to create a configuration (a kind of class path) just for running the Antlr Tool class. The new task finds the grammar files and feeds them to the tool. We also thread the output of the tool as a search path for the main Java compilation task. Finally, we define the inputs and outputs for the task, so that Gradle can decide whether it is necessary to even run the task.

Part of the fun of Gradle is that it is still a Groovy script, so there's a familiar and uniform syntax to defining variables and doing other non-declarative things, such as building up the list of grammar files for the Tool.

As you might guess from some of the comments, this is something of a first pass; the Maven plugin was a bit better at assembling the list of input file names in such a way that the Antlr3 Tool class knew where to write the output Java source files properly; if Tapestry used a number of grammars in a number of different locations, the solution above would be insufficient. It also seems roundabout to use Ant to launch a Java application ... I didn't see an easier way (though I have no doubt its hidden inside the Gradle documentation).

My experience getting this working was mostly positive; there's a very large amount of documentation for Gradle that helped, though it can be a bit daunting, as the information you need is often scattered across a mix of the Gradle DSL reference, the User Guide, the Javadoc and the GroovyDoc. Too often, it feels like a solution is only understandable once finished, working backwards from some internal details of Gradle (such as which exact classes it chooses to instantiate in a given situation) back through the various interfaces, Java classes, and Groovy MetaObject extensions to those classes.

In fact, key parts of what I did ultimately accomplish were discovered through web searches, not in the documentation. But, that also means that the system works.

Of course, this is the pot calling the kettle black ... one criticism of Tapestry can be paraphrased as we can customize it to do anything, and in just a few lines of code, but it can take three days to figure out where those lines of code go.

At the end of the day, I'm much happier with Gradle; the build process is faster, the build scripts are tiny and much, much easier to maintain, and the feedback from the tool is excellent. There's still many more issues to work out ... mostly in terms of Apache and Maven infrastructure:

• Ensuring the Maven artifacts are created properly, with the right dependencies in the generated pom.xml
• Generating a Maven archetype using Gradle
• Generating JavaDoc and Tapestry component documentation with Gradle, along with a minimal amount of pages to link it together (akin to the Maven site plugin)
• Generating source and binary artifacts and getting everything uploaded to the Apache Nexus properly

Regardless, I think all of these things will come together in good time. I'm not going back, and dearly hope to never use Maven again!

Better Namespacing in JavaScript

In my previous post, I discussed some upcoming changes in Tapestry's client-side JavaScript. Here we're going to dive a little deep on an important part of the overall package: using namespaces to keep client-side JavaScript from conflicting.

I'm not claiming to originate these ideas; they have been in use, in some variations, for several years on pages throughout the web.

Much as with Tapestry's Java code, it is high time that there is a distinction between public JavaScript functions and private, internal functions. I've come to embrace modular JavaScript namespacing.

One of the challenges of JavaScript is namespacing: unless you go to some measures, every var and function you define gets attached to the global window object. This can lead to name collisions ... hilarity ensues.

How do you avoid naming collisions? In Java you use packages ... but JavaScript doesn't have those. Instead, we define JavaScript objects to contain the variables and functions. Here's an example from Tapestry's built-in library:

Tapestry = {

  FORM_VALIDATE_EVENT : "tapestry:formvalidate",

  onDOMLoaded : function(callback) {
    document.observe("dom:loaded", callback);
  },

  ajaxRequest : function(url, options) {
    ...
  }, 

  ...
};

Obviously, just an edited excerpt ... but even here you can see the clumsy prototype for an abstraction layer. The limitation with this technique is two fold:

• Everything is public and visible. There's no private modifier, no way to hide things.
• You can't rely on using this to reference other properties in the same object, at least not inside event handler methods (where this is often the window object, rather than what you'd expect).

These problems can be addressed using a key feature of JavaScript: functions can have embedded variable and functions that are only visible inside that function. We can start to recode Tapestry as follows:

Tapestry = { 
    FORM_VALIDATE_EVENT : "tapestry:formvalidate"
};

function initializeTapestry() {
  var aPrivateVariable = 0;

  function aPrivateFunction() { }

  Tapestry.onDOMLoaded = function(callback) {
      document.observe("dom:loaded", callback);
  };

  Tapestry.ajaxRequest = function(url, options) {
    ...
  };
}

initializeTapestry();

Due to the rules of JavaScript closures, aPrivateVariable and aPrivateFunction() can be referenced from the other functions with no need for the this prefix; they are simply values that are in scope. And they are only in scope to functions defined inside the initializeTapestry() function.

Further, there's no longer the normal wierdness with the this keyword. In this style of coding, this is no longer relevant, or used. Event handling functions have access to variables and other functions via scoping rules, not through the this variable, so it no longer matters that this is often not what you'd expect ... and none of the nonsense about binding this back to the expected object that you see in Prototype and elsewhere. Again, this is a more purely functional style of JavaScript programming.

Often you'll see the function definition and evaluation rolled together:

Tapestry = { 
    FORM_VALIDATE_EVENT : "tapestry:formvalidate"
};

(function() {
  var aPrivateVariable = 0;

  function aPrivateFunction() { }

  Tapestry.onDOMLoaded = function(callback) {
      document.observe("dom:loaded", callback);
  };

  Tapestry.ajaxRequest = function(url, options) {
    ...
  };
})();

That's more succinct, but not necessarily more readable. I've been prototyping a modest improvement in TapX, that will likely be migrated over to Tapestry 5.3.

Tapx = {

  extend : function(destination, source) {
    if (Object.isFunction(source))
      source = source();

    Object.extend(destination, source);
  },
  
  extendInitializer : function(source) {
    this.extend(Tapestry.Initializer, source);
  }
}

This function, Tapx.extend() is used to modify an existing namespace object. It is passed a function that returns an object; the function is invoked and the properties of the returned object are copied onto the destintation namespace object (the implementation of extend() is currently based on utilities from Prototype, but that will change). Very commonly, it is Tapestry.Initializer that needs to be extended, to support initialization for a Tapestry component.

Tapx.extendInitializer(function() {

  function doAnimate(element) {
    ...
  }

  function animateRevealChildren(element) {
    $(element).addClassName("tx-tree-expanded");

    doAnimate(element);
  }

  function animateHideChildren(element) {
    $(element).removeClassName("tx-tree-expanded");

    doAnimate(element);
  }

  function initializer(spec) {
    ...
  }

  return {
    tapxTreeNode : initializer
  };
});

This time, the function defines internal functions doAnimate(), animateRevealChildren(), animateHideChildren() and initializer(). It bundles up initializer() at the end, exposing it to the rest of the world as Tapestry.Initializer.tapxTreeNode.

This is the pattern going forward as Tapestry's tapestry.js library is rewritten ... but the basic technique is applicable to any JavaScript application where lots of seperate JavaScript files need to be combined together.

Rethinking JavaScript in Tapestry 5.3

I've always had a love/hate relationship with JavaScript; some of the earliest motivations for Tapestry was to "encapsulate that ugly JavaScript stuff so I don't have to worry about it again." However, as I've come to appreciate JavaScript, over time, as a powerful functional language, and not as an incompletely implemented object oriented language, my revulsion for the language has disappeared ... even reversed.

Back around 2006, I started adding the client-side JavaScript features to Tapestry 5; this started with client-side form field validation, and grew to include a number of more sophisticated components. The good news is these features and components are fully encapsulated: they can be used freely throughout at Tapestry application without even knowing JavaScript. Tapestry includes the libraries (and related CSS documents) as needed, and encapsulates the necessary initialization JavaScript. The APIs for this were revamped a bit in Tapestry 5.2, but the core concept is unchanged.

The bad news is that the client-side is directly linked to Prototype and Scriptaculous (which are bundled right inside the Tapestry JAR file). These were great choices back in 2006, when jQuery was new and undocumented (or so my quite fallible memory serves). It seemed safe to follow Rails. Now, of course, jQuery rules the world. I've been talking for a couple of years about introducing an abstraction layer to break down the Prototype/Scriptaculous dependency; meanwhile I've recently seen that Rails and Grails are themselves moving to jQuery.

However, that abstraction layer is still important; I have clients that like MooTools; I have clients that are using YUI and ExtJS.

Certainly, it would have been too ambitious to try to start with such an abstraction layer from day 1. At the time, I had no real idea what the relationship between JavaScript on the client, and the application running on the server, would look like. Also, my JavaScript skills in 2006 are a fraction of what they are now. With several years of coding complex JavaScript and Ajax components for Tapestry, for TapX, and for clients, I think I have a much better understanding of what the APIs and abstraction layers should look like.

So suddenly, I have a number of goals:

• Allow Tapestry to work on top any JavaScript framework
• Support Prototype/Scriptaculous and jQuery as substrate frameworks "out of the box"
• Make the built-in Tapestry library first class: documented and release-on-release compatible
• Keep backwards compatibility to Tapestry 5.2

What I'm proposing is a gradual transition, over Tapestry 5.3 and 5.4, where new, documented, stable JavaScript APIs are introduced. and Tapestry and 3rd party libraries can code to the new APIs rather than to Prototype/Scriptaculous. The goal is that, eventually, it will be possible to switch the default substrate from Prototype/Scriptaculous over to jQuery.

Hibernate w/ transient objects

Am I missing something with Hibernate, or is it pretty darn hard to mix the following:

• Session-per-request processing (the approach provided by the tapestry-hibernate module)
• Transient objects (a wizard where a complex object is "built" across multiple request/response cycles)
• Persistent objects (the transient keeps references to some persistent objects)

Hibernate seems to make it a bit tricky for me here. I get a lot of odd exceptions, because the new object has references and collections that ultimately point to persistent objects that are detached (their session is long gone).

I'm having to write a lot of code to reattach dependencies, just before I render the page (which traverses the transient object, eventually hitting persistent objects) and before persisting the transient object.

I'm having to iterate over various collections and a few fields and lock the object, to convert it back to a persistent object from a transient one:

    public static void reattach(Session session, Object transientObject) {
        if (transientObject != null) {
            session.buildLockRequest(LockOptions.NONE).lock(transientObject);
        }
    }

In other cases, where the transient object has a reference to an object that may already be present in the Session, I must use code like:

  category = (Session) session.get(Category.class, session.getId());

If Tapestry supported it, I suppose some of this would go away if we used a long-running Session that persisted between requests. However, that has its own set of problems, such as coordinating the lifecycle of such a session (when is it started? When is it discarded? What about in a cluster?)

My current solution feels kludgey, and not like Idiomatic Java, more like appeasing the API Gods. I'd really like to see this happen more automatically or transparently ... for instance, when persisting a transient instance, there should be a way for Hibernate to "gloss over" these detached objects and just do what I want. Perhaps its there and I'm missing it?

I recently stumbled across a blog post by Kalle Korhonen: Why Tapestry?. Kalle has been very busy with Tapestry as the force behind Tynamo, a RAD toolkit built on top of Tapestry.

On the subject of performance:

the performance of the framework itself, both in terms of CPU and memory consumption, is simply phenomenal. Performance matters.

On how Tapestry compares with the competition:

What I really like to give as an answer to people who ask why one should use Tapestry is this: because it is well-balanced and comprehensive.{excerpt} There are a lot of other web frameworks that are optimized with a certain thing in mind and in that narrow field, they typically beat the competition. It's difficult though to be a good all-around contender but that's exactly what Tapestry is all about.

On the effectiveness of Tapestry as a solution:

Today's Java is far from your grandfather's Java a few years back and Tapestry makes the best use of the more advanced, modern JVM techniques available today, such as bytecode manipulation, annotation-based meta programming and introspection without reflection. Tapestry code is purposefully remarkably succinct.

On how Tapestry enables modularity:

Perhaps we've gone a bit overboard with modularity, but since it's just that simple with Tapestry, most of our modules are independently usable but seamlessly work together in the same web application as soon as you add them to the classpath.

There's quite a bit more, and it is both favorable to Tapestry and well balanced. Read the full posting.