Tapestry 4.0 Real Soon Now

It's been an interesting week. First off, I had to leave my brand new apartment in Portland, OR after one night to come down to San Francisco for a three-day intensive Tapestry training session. Teaching Tapestry to twenty people in three days is a bit of a challenge, but the results were very well received; I understand that all new web development at shopping.com will be done using Tapestry.

Meanwhile, in the struggle to get Tapestry 4.0 out, the first vote for a final 4.0 release failed, with a veto by Kent Tong. At issue was the gaps in the documentation, particularily the documentation related to validation. This prompted new contributer Jesse Kuhnert to fill in the gaps there, so I'm in the process of producing a new release candidate, 4.0-rc-2, which will be up on the mirrors sometime tomorrow.

This does show that the process works ... but also underscores that the way many other projects function might not be a bad idea. Apache will generally release numbered releases at intervals, and after the fact elevate a release to "final, stable" status. Had we been following that scheme, we'd be at, say, 4.0.15 around now. If we vote that as a stable release ... well, we don't have to do any extra work beyond publicising it; it will already have been available, presumably, for a week or two minimum before such a vote.

On another front, I've been using Maven 2.0, as I try to rework HiveMind to build under Maven instead of Ant. For the most part, Maven 2 feels more like a finished work than Maven 1 ever did. I haven't been wringing my hands in frustration the same way. I just now downloaded the 2.0.1 release and will see if that fixes the few issues I have been encountering ... such as reports not been run and integrated into the site output.

This does tie together ... once HiveMind and then Tapestry are building under Maven, even having regular point releases will be much less important, since it will be quite easy for the average developer to pull down the source and build. Alas, Tapestry and HiveMind currently raise the bar for that a little too high for the casual developer.

Has anyone gone through the process of converting Forrest XML documentation to Maven XML documentation? They are similar is concept but slightly different in details; I'd like to save myself the effort of writing conversion scripts and stylesheets.

Moved to Portland

Well, now we've gone and done it. My wife and I have sold our house outside Boston and relocated to Portland, Oregon. Time to start over in an entirely different part of the country, one that seems to align better with our tastes, politics, and interests. Given the people we met at OSCON 2005 (also in Portland), I know there's a good community of open-source and Java developers around here ... I can't wait to start meeting them!

Two New Tapestry Committers: Jesse Kuhnert and Kent Tong

The Tapestry project has just added two new committers!

• Jesse Kuhnert - Very active with the Tacos component library, which is the hub for Ajax components for Tapestry. He's got an inside line on what it'll take to make Tapestry the best framework for Ajax development.
• Kent Tong - Active mentor on the user and developer mailing lists, Kent has been keeping me and the other Tapestry committers honest for quite a while now ... and he's written a great book on Tapestry 4.

Great things are afoot for Tapestry ... fresh blood, a final 4.0 release very, very soon now, and an increasing number of high profile sites in the pipeline (and no, I can't say more).

Tapestry @ JavaForge

I'm just about done setting up Tapestry @ JavaForge, an auxillariy site for Tapestry components and extensions. I've been experimenting with Maven 2, so far with some decent success. I especially like the Wiki-ish "almost plain text" format for documentation. Maven 2 still has a number of teething issues, and I haven't even touched the critical multi-project support yet, but I'm cautiously optimistic.

No downloads just yet, but the source code is easily accessible via Subversion and builds using Maven, which is close enough.

Can you tell I'd really rather not be balancing my checkbook right now? Merry Turkey-Day!

No Tapestry at ApacheCon 2005

Hope I'm not disappointing too many people, but I will not be able to do the Tapestry Tutorials at ApacheCon on Dec 10th, as previously stated. They didn't get enough tutorial sign ups by their Nov. 12th deadline (ApacheCon does terrible marketing), and since I really hadn't wanted to do a tutorial in the first place, I offered to cancel rather than try and come up with a last-minute way to boost attendance.

I was somewhat looking forward to this, since I haven't done any speaking engagements in too long. But with Tapestry 4.0 in the final stages, plus the fact that, as I type this, I'm supposed to be stuffing books into boxes for my move to Portland, Oregon next week ... well, let's just say that skipping ApacheCon goes a small way towards easing my monumental stress level.

Woops! Killed JavaForge.com

I think I may have killed JavaForge. No, really. It seems to have blown up and crashed hard as I was uploading the tapestry-prop site tarball into their documenent management system.

I really want to like JavaForge because it is free, and because it allows Subversion hosting ... but any instability (it's been down for about 30 minutes now) is troubling.

And Apache's JIRA is down again ...

... and I was going to fix all those outstanding Tapestry bugs tonight. :-) But I can't. Sigh.

Documentation for Script Templates

We're in the final stages of Tapestry 4.0, trying to fix the most glaring documentation ommissions, and tidy up the bug list.

I've finally written documentation for the script template specification DTD, which is a very important file with the growing emphasis on JavaScript with web applications.

Tapestry has had, for a few years, some very advanced code for generating dynamic JavaScript to complement the dynamic HTML. It's centered in these script templates that take care of organizing the JavaScript into two large blocks (one at the top of the page, the other at the bottom) ... which is better than having it scattered about. As importantly, the JavaScript generation understands the need for uniqueness, to prevent name collisions in client-side variables and functions. This is the infrastructure that lets clever folks, like the ones at Tacos, create Ajaxian components that just work without any special attention by the developer ... regardless of whether there's just one such component, or dozens, on a page. Tapestry has the infrastructure to support this, with out requiring any special configuration of servlets or anything else.

Improving Tapestry performance

I just spent the week with a high profile client that is interested in potentially using Tapestry for a very large scale site ... millions of hits per hour. Their in-house framework is quite capable of operating at this scale, through a combination of draconian restrictions on database access and server-side state, and a total avoidance of any kind of reflective object access. These are people who literally cannot give an inch on performance.

One of the ideas that bounced around was something promised for some future release of OGNL: bytecode enhancement. That is, in some cases, OGNL 3 is expected to identify places where it can create a class on the fly to expedite access to a property, rather than always relying on reflective access as it does today.

Alas, that hasn't happened yet, and Tapestry is still using OGNL 2.6.7.

But, I thought, what if we created a new binding prefix to use instead of OGNL, for this purpose. Because of HiveMind, this approach can be packaged seperately from the framework proper, and plug right in.

... and it works. I built a little peformance test harness and tried to figure out how many nanoseconds it takes to perform an operation; an operation involves a read and then an update. Here's one of the operations from the harness:

        Op op = new Op()
        {
            public void run(PropertyAccessor accessor)
            {
                Long value = (Long) accessor.readProperty();

                long primitive = value.longValue();

                accessor.writeProperty(new Long(primitive + 1));
            }
        };

The PropertyAccessor object is either created from bytecode, or implemented using OGNL (so that we can make the comparisons).

I did a number of test runs, with a number of operations:

              10000 iterations |  Direct ns |    OGNL ns
------------------------------ | ---------- | ----------
                 name - warmup |    4288.00 |  847364.00
                          name |    1777.00 |   18426.00
                  int - warmup |    2891.00 |   81127.00
                           int |     838.00 |    7497.00
                 long - warmup |    2969.00 |   28207.00
                          long |     617.00 |    7256.00

             100000 iterations |  Direct ns |    OGNL ns
------------------------------ | ---------- | ----------
                 name - warmup |    4282.00 |  819634.00
                          name |     972.00 |    7527.00
                  int - warmup |    2947.00 |   74425.00
                           int |     242.00 |    5955.00
                 long - warmup |    2910.00 |   27492.00
                          long |     209.00 |    6046.00

             500000 iterations |  Direct ns |    OGNL ns
------------------------------ | ---------- | ----------
                 name - warmup |    4182.00 |  852125.00
                          name |     857.00 |    6756.00
                  int - warmup |    2958.00 |   81820.00
                           int |     170.00 |    5724.00
                 long - warmup |    2793.00 |   34990.00
                          long |     215.00 |    5785.00

          2,000,000 iterations |  Direct ns |    OGNL ns
------------------------------ | ---------- | ----------
                 name - warmup |    4251.00 |  843185.00
                          name |     823.00 |    6553.00
                  int - warmup |    2927.00 |   48788.00
                           int |     144.00 |    5799.00
                 long - warmup |    2961.00 |   34945.00
                          long |     180.00 |    6173.00

The results show that the direct access is around 10x faster than reflective access, which is in line with the general documentation about reflection in JDK 1.5. Still, I'm troubled that the cost per operation seems to continue going down as the number of operations increases. This could be the effect of hotspot (though the elapsed time seems short for hotspot to get very involved) ... or it could represent a problem in my performance test fixture.

To use this, you just use the prefix "prop:" instead of "ognl:". And, of course, it only works for simple properties, not property paths or the full kind of expressions used in OGNL.

I'm hosting the code on JavaForge and will make some kind of release available soon. Perhaps it will migrate into the framework proper at some point.

Further Down The Trail

Chris Nelson has published his second article on Trails: Further Down the Trail.

Seems like a lot of people are getting excited about Trails, and rightly so. I can see a large number of projects getting built this way. It's a great idea, it's developer-focused, it gets you great results fast. I've also seen Chris mention integrating Ajax components from Tacos. What's easier than using Tacos components for Ajax behavior ... using Trails and getting the Ajax stuff for free!

By the way, one of the most remarkable aspects of Trails is that Chris has been building Trails as a way to learn Tapestry. Either Chris is really some kind of comic-book alien machine super-intelligence (unlikely, I've met him in person), or Tapestry isn't quite as hard to learn as some people seem to think. Perhaps Chris had less Struts and JSP clutter to unlearn.

A cool 1500

Just finishing up a fix for TAPESTRY-602 and, in the process, added the 1500th test to the Tapestry test suite.

Even Easier Tapestry Examples

I took up an idea posted on the Tapestry User mailing list and, starting with beta-12, the Tapestry examples will be even simpler to obtain and run.

The new Tapestry examples distribution will be a slice of a JBoss distribution, with the Workbench and Virtual Library deployed into it. Yep, about 99% JBoss and about 1% Tapestry, but it means you can untar the examples distro and execute run.bat or run.sh and have it all up and running in less than a minute.

Once again, the less enlightened will bitch and moan about using JBoss. And once again folks, this is a demo. Everyone has their favorite container and I've gotten annoying flak in the past for not supporting all of them ... as if that was the point. In truth, the point is to get an easy to run demo. Perhaps someday I'll use Geronimo, because it is also an Apache project. But I go a ways back with JBoss, so it was easy for me to build and package, and is a good use of my very scarce time.

There is NO dependency between Tapestry and JBoss. Deploy wherever you want, but if you want a quick turnkey demo, download the demos.

Why I like Annotations

I've been working with JDK 1.5 annotations for several months now with exceptional results. Perhaps it's just a reflection of Tapestry, which has grown to use a pattern of design and development I've come to call composite coding.

Composite coding is a kind of intersection point between traditional class-oriented inheritance, aspect oriented programming, and code generation. This approach was already in place nearly two years ago, in Tapestry 3.0. When you define a class as abstract, and define transient and persistent properties (using XML) in Tapestry 3 what you are really doing is compositing your code with code generated by Tapestry.

This is not so much different from traditional approaches, where we composite the code we write with code from base classes often written by others. That's the nature of inheritance (and whether that is a good model for creating a framework is a discussion for another day).

Aspect oriented programming is also a way of compositing different sets of code together to form the final class definitions used at runtime. In fact, aspect oriented programming is such a general term that it easily encompasses what Tapestry is doing ... in fact, the uses of composite coding in Tapestry are very much within the domain of aspect oriented programming: cross cutting concerns related to page pooling and server-side state managements.

What I like about annotations, as applied to Tapestry 4, is that it integrates the code with the composite operations to be applied to that code ... that is, to make a property persistent, add the @Persist annotation directly to the method, rather than configure persistence elsewhere (in Tapestry 3, in a companion XML file).

One of the complaints of this approach is that it puts configuration data directly inside Java class files, where changing configuration requires a recompile and re-deploy of the application.

That's a pretty lame complaint and, to my ears, is an attempt to resist change. Sorry folks, get used to embracing change ... or get used to saying "want fries with that?"

Even in an extreme example, say the EJB3 @Table annotation, the problem isn't with the annotations ... it's with the framework that utilizes those annotations. In many cases, an annotation sets a default value ... one that may need to be overridden to match a particular deployment configuration.

In fact, this is only one step removed from the mess that was EJB 1 and 2 ... where the mythical application deployer role was responsible for configuring and deploying EJBs by configuring deployment details such as ... databases and tables. Sun dropped to ball on describing how this person did their job. In fact, the only effective way would be to explode EARs, including nested WARs and JARs, so that the ejb-jar.xml configuration files (and their many friends) could be edited, then repackage everything.

In my view, BEA, IBM and JBoss (in fact, the whole application server sector) dropped the ball by NOT providing a mechanism to override the defaults specified inside the ejb-jar.xml files short of exploding and repackaging.

Annotations are no different; tools and frameworks that rely on annotations should take into account the fact that deployment may need to change some of the configuration specified in the annotations. In other words, don't work directly from the code annotations, construct a model from the annotations, but allow other vectors for controlling that model before it is acted upon.

Yes, this is the model used in Tapestry 4, where an optional specification is parsed into memory, then modified to reflect information gleaned from annotations.

One of the major things to admire about the Ruby community is the emphasis on The Code Is The Thing. Nobody gives a sneeze at all the meta-programming in Ruby, because, in the end, they still have the standard ways to read and update Ruby object properties. JDK 1.5 Annotations are as close as Java currently gets to meta-programming and it brings the emphasis back to The Code, where it belongs.

In the past, I often was asked why Tapestry used so much XML. "Because it's convenient" was my response ... a convenient way to store some rich, hierarchical data. Well, annotations are even more convenient, and I think, more in the spirit of how we should be coding.

Apress book on Tapestry

Just spotted this forthcoming book on Apress's web site: Beginning Open Source Enterprise Java: Spring, Hibernate, JBoss and Tapestry

Beginning Open Source Enterprise Java takes you through the construction of a complex enterprise Java application centered around JBoss, Spring, Hibernate, Tapestry, Ant, and other supporting tools for development and testing. This book is ideal if you’re new to Open Source Java, and want to build enterprise Java applications from scratch, using the full range of available Open Source tools and frameworks.

This book features the most successful and prevalent Open Source tools, along with some lightweight frameworks and tools. You’ll learn how to build a complete enterprise application, how to integrate the different Open Source frameworks to achieve this goal, and techniques for rapidly developing such applications.

Virtual Library Renovations

The Virtual Library example application has now been completely renovated for Tapestry 4.0. It now looks like an application built for Tapestry 4.0 from scratch. It makes heavy use of annotations, friendly URLs (including some custom friendly URLs), application HiveMind services ... all the 4.0 goodies. The only thing that didn't change was the underlying EJBs (except to switch over to HSQL from McKoi).

I'll be writing up some comparison documentation for the next beta.

One of the areas I'm really happy with is the inheritance hierarchy; I was able to flatten it quite a bit. The old code had several levels of inheritance, centered around the pageValidate() method used to restrict portions of the site to logged in users, or to users with admin privilege. The new system moves all that logic to a single base class and drives it by @Meta annotations (with some subclasses overriding the @Meta from the base class).

Part of the transformation was the removal of the VirtualLibraryEngine; all the logic that used to be there is now in various HiveMind services. I borrowed a page from Spring and created a RemoteTemplate service that can execute a RemoteCallback (with retry logic for RemoteExceptions). That removed lots of very ugly loop-and-retry logic from many pages and components, and replaced it with less-ugly RemoteCallback inner classes.

I hope people will take a close look at the new Virtual Library; it continues to be a small, manageable application that can serve as an excellent template for building much more ambitious, real-world projects.

Hopefully in the next beta release, these examples will be distributed as a pre-configured JBoss installation, rather than a patch on top of JBoss. This will be bigger but even easier to setup and execute. Probably in 4.1 we'll shift the code around to use JPox, Cayenne or Hibernate and get away from requiring EJBs or an application server.

Blogs Aren't For Support

Hate to say this, but this Blog is not a forum for getting free Tapestry support. I've been seeing a creeping number of requests for help and this just isn't the right place!

For free community support, subscribe to the Tapestry User Mailing List and learn how to ask a question that will get a response.

For professional support, you can contact me, or any other member of the Tapestry Support Network to purchase support.

Google: Tapestry

Just did a ego gratification Google search on the word Tapestry. Guess what's on the top of the list? Used to be, you had to qualify the search as "java tapestry" to even get close. Does this signal any new level of adoption of Tapestry?

In a perhaps related note, I got my Manning royalty statement for Tapestry In Action for April 1 - June 30th ... and it's my biggest (by 50%) to date. Perhaps that has something to do with my appearance at JavaOne?

ApacheCon 2005 - Tapestry Tutorial

I'll be presenting a half-day tutorial on Tapestry at this year's ApacheCon -- an afternoon session on Saturday, Dec 10. This is a great chance to learn a lot about Tapestry and Tapestry 4.0 especially ... I'll be highlighting new features, such as annoations and the improved validation framework. The format will be based on my labs, though I don't think the time constraints will support interactive labs; instead, I'll be doing the work as we go, explaining Tapestry and Tapestry concepts while getting various mini-applications running in Tapestry.

ApacheCon is in San Diego this year, which offers fewer distractions than Las Vegas, but will be a pleasant change nonetheless.

Ouch! Class.getDeclaredMethods() works differently in different JDKs

This one is a bit of a pain. In Tapestry, your page and component classes tend to be abstract, with Tapestry providing a subclass, filling in additional methods, fields, constructors and implemented interfaces.

Because of that, it's necessary to duplicate the checks normally done by the compiler ... to check that all abstract methods inherited from base classes actually end up with implementations in the enhanced subclass (that is, the subclass that Tapestry creates).

At the core of this is code that finds all the non-abstract methods in the enhanced class, and up its inheritance chain. It then checks all abstract methods from superclasses or implemented interfaces to ensure that each one is, in fact, implemented.

The problem here is that the built-in Eclipse compiler seems to work differently than the Sun JDK compiler (this is for JDK 1.5).

Under Eclipse, an abstract class that implements an interface does not report those interface methods from getDeclaredMethods(). The only methods reported are the ones in the actual code for the class.

Under Sun JDK, an abstract class will report those interface methods, even if it does not provide implementations for them.

Now, I have tests for this class that I usually run from Eclipse. In fact, this was triggered by a specific bug concerning methods inherited from interfaces; the code is broken into two parts (scanning the inheritance tree for implemented methods, and scanning the interfaces for missing methods). Under Eclipse, interface methods are not reported from getDeclaredMethods(), so we make it to phase 2.

Under Sun JDK, interface methods are reported, it looks like a non-interface abstract method with no implementation, and we fail in phase 1 -- breaking my unit test.

Sigh. This may take some hackery to work in Eclipse and in Sun JDK.

Reworking Tapestry Docs and Demos

A slight shift of focus this week, especially today. I've been reorganizing the Tapestry project home page; from now on, the menu down the left side will be consistent from page to page, not contextual. The tabs across the top are now more like book marks. I think it will make navigating the site easier. Further, the first thing on the home page is another set of quick links to the most common pages, including the Users Guide, Quick Start Tutorial, JavaDoc, etc.

Now, the Quick Start tutorials represent another way to ease newbies into Tapestry. It's been something I delegated to others in the past, and it never happened, so I've taken it back on myself. I'd rather be writing new code or even fixing bugs (or working on my own labs), but I've been getting feedback that the QuickStart is making a big difference among prospective Tapestry users. In fact, there have been over 2200 downloads of the tutorials already!

In the same light is my new approach to the Tapestry demos: the Workbench and the Virtual Library. In the past, you had to build them yourself (because they use non-ASF code, and can't be distributed from the ASF). But that's asking a lot from people, especially those just casually investigating Tapestry.

The new approach is a seperate distribution for the demos, that comes from a non-ASF server (http://howardlewisship.com/downloads/quick-start/).

This bypasses the concerns of the ASF ... though it does eat into my personal web site's bandwidth! But what it means is people can get up and running with Tapestry really easily:

• Install JBoss 4.0.2
• Install Ant
• Unpack the examples
• Run the build script using Ant
• (Re)start JBoss
• Run the applications in a browser

A step up from this would be an installer, but I was short on time to do that today. It would be nice, and there are reasonable, free installers out there. That would eliminate the need for Ant ... but I assumed that nearly everyone interested in Tapestry would already have Ant.

It's a shame that living inside Apache causes these convulsions; it will rise up again in 4.1 when we start wanting to do deeper integration of Tapestry and existing AJAX client-side JavaScript libraries ... that work will have to take place largely off-Apache.