Twitku

I just noticed that Ted Neward is on Twitter ... but only twits in Haiku form.

Maven for dependencies, not building

I've been working a bit with Ivy but finding it doesn't quite fit my needs (or that I have a lot more to learn). What I've found with Ivy is that it has its way of doing things, and part of The One True Path is that you should create a repository just for your organization ... they really hate the idea of depending on the central Maven repository.

In any case, my need is to make it easy to build Tapestry or Tapestry demos from source without a lot of fuss. That's where I like Maven's dependency management, especially for the most common artifacts.

The other problem I've had with Ant is that it doesn't get Maven scopes perfectly, or Maven artifacts (such as source JARs) without a lot of redundant typing.

So, for the mean time, I'm back to Ant plus the Maven Ant tasks.

What I want is to create and manage four folders:

• runtime
• runtime-src
• test
• test-src

The main folders, runtime and test, contain JARs needed for compilation/execution of the production code, and for the test code. This could expand to include more of Maven's scopes in the future, such as provided, but it suits my current needs. The -src folders contain source JARs, which I find essential to being productive. Nothing irks me more than opening a base class or interface and getting that ugly view of just the method names because source isn't available. It gives me flashbacks to miserable sessions with WebLogic trying to figure out what the hell it was doing.

So, how can we get there without the rest of Maven? How about this:

<?xml version="1.0"?>
<project name="common" xmlns:mvn="urn:maven-artifact-ant">

    <!-- Names of common directories used in the build. -->

    <!-- Directory in which temporary files are created. -->
    <property name="target.dir" value="target"/>
    <property name="target.lib.dir" value="target/lib"/>

    <!-- Directory to which imported dependencies are placed. Four subfolders are created: runtime, runtime-src, test
         and test-src. -->
    <property name="lib.dir" value="lib"/>

    <path id="maven-ant-tasks.classpath" path="maven-ant-tasks-2.0.9.jar"/>

    <typedef resource="org/apache/maven/artifact/ant/antlib.xml" uri="urn:maven-artifact-ant"
             classpathref="maven-ant-tasks.classpath"/>


    <macrodef name="import-dependencies">
        <attribute name="scope"/>
        <attribute name="folder" default="@{scope}"/>
        <element name="dependencies" implicit="true"/>

        <!-- locals -->
        <attribute name="lib" default="${lib.dir}/@{folder}"/>
        <attribute name="lib-src" default="@{lib}-src"/>

        <attribute name="target-lib" default="${target.lib.dir}/@{folder}"/>
        <attribute name="target-lib-src" default="${target.lib.dir}/@{folder}-src"/>

        <attribute name="lib-fileset" default="@{scope}.dependency.fileset"/>
        <attribute name="src-fileset" default="@{scope}.dependency.sources.fileset"/>

        <sequential>
            <mvn:dependencies pomrefid="project.pom" verbose="true" usescope="@{scope}"
                              filesetid="@{lib-fileset}"
                              sourcesfilesetid="@{src-fileset}">
                <remoteRepository id="tapestry-nightly"
                                  url="http://tapestry.formos.com/maven-snapshot-repository">
                    <snapshots enabled="true"/>
                </remoteRepository>
                <remoteRepository id="maven-central"
                                  url=" http://repo1.maven.org/maven2"/>
                <remoterepository id="openqa-release"
                                  url="http://archiva.openqa.org/repository/releases/"/>
            </mvn:dependencies>

            <mkdir dir="@{target-lib}"/>
            <mkdir dir="@{target-lib-src}"/>
            <!-- Flatten them, so we can do a sync. -->
            <copy todir="@{target-lib}" flatten="true" preservelastmodified="true">
                <fileset refid="@{lib-fileset}"/>
            </copy>

            <copy todir="@{target-lib-src}" flatten="true" preservelastmodified="true">
                <fileset refid="@{src-fileset}"/>
            </copy>


        </sequential>
    </macrodef>

    <macrodef name="remove-overlap">
        <attribute name="source"/>
        <attribute name="target"/>

        <attribute name="fileset.property" default="@{source}.overlap.files"/>

        <sequential>
            <pathconvert pathsep="," property="@{fileset.property}">
                <fileset dir="@{source}"/>
                <flattenmapper/>
            </pathconvert>

            <delete dir="@{target}" includes="${@{fileset.property}}"/>
        </sequential>
    </macrodef>

    <target name="setup-libs" description="Copy dependencies to lib folder.">

        <mvn:pom file="pom.xml" id="project.pom"/>

        <!-- Delete the scratchpad space. -->
        <delete dir="${target.lib.dir}" quiet="true"/>

        <import-dependencies scope="runtime"/>

        <!-- For the moment, this is somewhat broken: test scope is a super-set of compile/runtime scope.
             Everything in the runtime folders will end up in the test folders, plus more. -->
        <import-dependencies scope="test"/>


        <!-- Snapshots come down with the datestamp/version number, we need to fix that. -->
        <move todir="${target.lib.dir}" preservelastmodified="true">
            <fileset dir="${target.lib.dir}"/>
            <!-- Turn the date/version stamp back into SNAPSHOT -->
            <regexpmapper from="^(.*)(\d{8}\.\d{6}\-\d+)(.*)$$" to="\1SNAPSHOT\3"/>
        </move>

        <!-- Delete the overlap between lib/runtime and lib/test -->

        <remove-overlap source="${target.lib.dir}/runtime" target="${target.lib.dir}/test"/>
        <remove-overlap source="${target.lib.dir}/runtime-src" target="${target.lib.dir}/test-src"/>

        <echo>*** Synchronizing ${lib.dir} ...</echo>
        <sync todir="${lib.dir}" verbose="true">
            <fileset dir="${target.lib.dir}"/>
        </sync>
        <echo>... done.</echo>

        <delete dir="${target.lib.dir}" quiet="true"/>
    </target>

</project>

Just add the Maven Ant tasks JAR and a pom.xml (that exists exclusively to identify dependencies) and you are off and running.

Some of my requirements are awkward to implement in Ant:

• Flattened directories; Maven really wants to build a mirror of the repository, with directories for groups, but I want everything in a single directory.
• Remove overlap; Maven's test scope includes the runtime scope, extra work is necessary to keep the test directory to just the additional JARs without redundantly including what's already in runtime
• Timestamps; I want to sync the directories, not just copy in, to account for version number changes and I don't want to change timestamps unless a file has changed (to keep the IDE from wasting time rescanning and reparsing it)

The script creates a temporary directory under target, which is first used to flatten the copy from the local repository, then to remove the runtime vs. test redundancies before sync-ing it over the proper lib directory.

The best part of all this? Nothing happens until ant setup-libs rather than the frequent surprises that Maven gives you!

Eventually, I'll repackage some of this into a common build.xml that can be shared across modules. In the meantime, it's good enough for demos and labs in the workshop.

Firefox 3 : The Good, the Bad

The Good is that it does feel faster,and it now sports a UI that looks much more Mac native.

The Bad is that many plugins don't work yet, such as FireBug! In addition, the Selenium support for Firefox is glitchy with the new release.

Update: People are coming out of the woodwork to point me at the Firebug for FF3 ... check the comments!

Groovy: Leaky Abstractions and Confusing Exceptions

Here's an odd place to get a ClassCastException:

• org.hibernate.type.StringType.toString(StringType.java:44)
• org.hibernate.type.NullableType.nullSafeToString(NullableType.java:93)
• org.hibernate.type.NullableType.nullSafeSet(NullableType.java:140)
• org.hibernate.type.NullableType.nullSafeSet(NullableType.java:116)
• org.hibernate.param.NamedParameterSpecification.bind(NamedParameterSpecification.java:38)
• org.hibernate.loader.hql.QueryLoader.bindParameterValues(QueryLoader.java:488)
• org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1554)
• org.hibernate.loader.Loader.doQuery(Loader.java:661)
• org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
• org.hibernate.loader.Loader.doList(Loader.java:2211)
• org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2095)
• org.hibernate.loader.Loader.list(Loader.java:2090)
• org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:375)
• org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:338)
• org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:172)
• org.hibernate.impl.SessionImpl.list(SessionImpl.java:1121)
• org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
• org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:811)
• sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
• sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
• sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
• java.lang.reflect.Method.invoke(Method.java:585)
• org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
• groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
• groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
• groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
• org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
• org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
• org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
• org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethod0(ScriptBytecodeAdapter.java:198)
• com.nfjs.hls.blog.services.AuthenticationManagerImpl.authenticate(AuthenticationManagerImpl.groovy:34)

That StringType object was expecting a string, but got a org.codehaus.groovy.runtime.DefaultGroovyMethods$2. Seems like a little bit of Groovy has leaked unexpectedly into my Java.

The context is a method for authenticating a user's handle and password:

   Blogger authenticate(String handle, String plaintextPassword)
    {
        def authenticationCode = computeAuthenticationCode(handle, plaintextPassword) 

        def query = session.createQuery("from Blogger where handle = :handle and authenticationCode = :code")

        query.setProperties([handle: handle, code: authenticationCode])

        def result = query.uniqueResult()

        if (result)
            asm.get(UserCredentials.class).userId = result.id;

        return result
    }

   def computeAuthenticationCode(handle, plaintextPassword)
    {
        def md = MessageDigest.getInstance("MD5")

        // Salt the digest with the lower-cased handle
        md.update(handle.toLowerCase().bytes)
        md.update(plaintextPassword.bytes)

        md.digest().encodeBase64()
    }

A little hunting around showed that the ClassCastException was related to the authentication code.

Now the encodeBase64() method should be returning a String, one would think. Ah, a little poking around in the Groovy JDK docs and in fact, it returns Writable. After a few experiments, I found that the following change worked and was most pleasing:

   md.digest().encodeBase64() as String

This is the exact kind of error that static typing is designed to catch. Certainly, this would be caught by some tests eventually (I'm working a little fast and loose here), but in the end I had to work with the debugger to chase down what was wrong.

And that brings up a more significant problem: it appears that in Groovy, local variables are not visible in the debugger. I suspect they may be stuck inside the metaClass object or somewhere else I haven't found yet. That is significantly less than ideal. Update: User error; there was a button in the debugger pain that made the local variables visible.

What I'm finding, working with Groovy, is that for the trivial kind of code I'm writing to support my application, any extra bang I get for using Groovy instead of Java is being offset by leaky abstractions, my own lack of familiarity with Groovy, and the very shakey developer tools (even inside IDEA!). One aspect of this is just how trivial the Java code for a Tapestry application is; there's just not much room for Groovy to improve on pure and simple Tapestry Java.

However, the exercise is certainly worth it, because many people will be using Groovy with Tapestry and it should work without any hickups.

JavaScript var vs. Groovy def

I keep typing the following:

var blogger = new Blogger()

when I should be saying

def blogger = new Blogger()

def is used to define new methods and variables. The problem is that the JavaScript syntax compiles ... it just doesn't execute. I looks like a call to method var().

The code fails at runtime with groovy.lang.MissingPropertyException: "No such property: blogger for class: com.nfjs.hls.blog.pages.Login".

Just takes a little getting used to. Perhaps this is where polyglot programming breaks down a little ... switching between too similar languages.

Using Groovy for IoC Service Decorators

Continuing with my experiments with Groovy, I took a pass at writing a service decorator in Groovy. In Tapestry, the RequestHandler service is the primary execution path; everything filters through it. It's an extensible service, based on the pipeline pattern, so I could contribute a filter into the service, but instead I wrote a decorator around the service.

    def decorateRequestHandler(service, Logger logger)
    {
        {request, response ->
            def start = System.nanoTime()

            def result = service.service(request, response)

            def elapsed = System.nanoTime() - start

            logger.info(String.format("Request time: %5.3f s -- %s", elapsed * 10E-9d, request.path))

            return result
        } as RequestHandler
    }

Tapestry's naming convention is that a module class method named decorateXXX is a decorator for service XXX. The service implementation is passed in. The return value must be the same type as the service (that is, implement the service's interface). Again, we use a closure in Groovy as an easy alternative to an inner class in Java. The code here logs the elapsed time (in seconds) for the request.

What's interesting is just how much gets into the stack trace:

 at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106)
 at $RequestHandler_11a6ead473a.service($RequestHandler_11a6ead473a.java)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
 at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
 at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
 at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
 at com.nfjs.hls.blog.services.AppModule$_decorateRequestHandler_closure2.doCall(AppModule.groovy:49)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
 at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:248)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
 at groovy.lang.Closure.call(Closure.java:292)
 at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:48)
 at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:72)
 at $Proxy147.service(Unknown Source)
 at $RequestHandler_11a6ead4732.service($RequestHandler_11a6ead4732.java)

In this view, $RequestHandler_11a6ead4732 is a Tapestry runtime fabricated class, the proxy for the service. $Proxy147 is a JDK dynamic proxy which is eventually hooked up to the closure code. The service parameter (to the decorateRequestHandler) is the instance of CheckForUpdatesFilter.

What looks like a simple method invocation turns into several levels of indirection through the Closure and MetaClass code, and then ultimately a reflective method invocation before it picks back up with the first filter in the pipeline (CheckForUpdatesFilter).

This is not especially troubling; I suspect it would be all but impossible to measure the "extra" time the Groovy code takes (vs. an equivalent Java implementation). Still, this could easily add up inside a tight loop, such as the rendering state machine built into Tapestry components. Certainly Merlyn was having performance issues with his visual processing application.

Real World Haskell

When I'm in the mood to feel really dense, I try to wrap my brain around functional programming. My poison of choice is Haskell, since that's the purest and least compromising of the languages, in my uninformed opinion. I have a couple of books on the subject and every once and a while I struggle with the syntax and the alternate mode of thought.

I recently stumbled across a very approachable on-line book-in-progress about Haskell: Real World Haskell.

I still have my doubts about functional programming; I should love it, because I tend to think about code and how code fits together all the time. However, the heavy mathematical bent of functional programming is a bit of a challenge, and the FP-ers seem to value conciseness over all other things. I worry that Haskell programs are write-only in the way that Perl can be; that every program is an elegant puzzle-box, but a puzzle nonetheless.

I'm still intrigued not due to the conciseness, or the promise of parallel execution, but because of lazyness. Having lazy evaluation built right into the language means that a lot of common coding conventions and patterns are simply a natural byproduct of the use of the language.

For example, throughout Tapsestry there are examples of code that is based on a recursive algorithm implemented non-recursively (typically, using queues). In Haskell, you get that for free pretty much universally: the Haskell execution stack does not get very deep, though there's a kind of stack of as-yet-unevaluated expressions (it looks like the Haskell folk call these "thunks") behind the scenes, acting like the queues I'm coding directly.

Because it's baked into the language, you are free to express algorithms very purely, without the implementations being obscured by performance details (such as converting recursion into queues). In fact, the laziness of Haskell is really good at handling infinite lists (as long as you don't ask for the length of that list!)

There's a bit of hype going on in the FP/Haskell/Erlang space. For example, people keep talking a lot about the need for FP to tackle large numbers of cores. Does the Emperor have any clothes here? I'm not certain: Haskell and Erlang use non-native threads. That's a great way to maximize a single core and all the no-sideffects, no-mutable state simplifies the problem vastly from a typical Java application.

However, Erlang typically approaches the highly scalable space by starting (and, as necessary, restarting) lots of processes. The immutable state must be extracted from one process and serialized to another; Erlang has a native format for this that's pretty efficient. Haskell adds similar functionality as an add-on library. But from my point of view, it doesn't feel that different from JMS and messaging with a bit of coding discipline.

In fact, the point of view of 1060 NetKernel is that you can have the advantages of an Erlang style model, but implement it in your choice of languages (Java, JavaScript, Python, etc., etc.). We may have a gig at Formos where I'll have a chance to really look into NetKernel, I've been hovering around the edges on it for a couple of years.

Time Breakdown of Modern Web Design

Tapestry with Groovy

While I'm waiting for the vote on Tapestry 5.0.12, I'm starting some work on a more complete demo application. As usual, I'm trying to teach myself a bunch of new things, such as using Ant and Ivy (in place of Maven), and using Groovy for my pages, entities and services.

I'm still having trouble with live class reloading in Tomcat, so I'm staying with Jetty for a bit longer.

The Groovy stuff for pages and such is just fine. My demo application is a simple blogging site (boring stuff, but a domain model everyone understands). I don't have much yet, but my first page lists recents blog postings:

<t:grid source="recentPostings"/>

And the page class is written in Groovy:

package com.nfjs.hls.blog.pages

import org.apache.tapestry5.ioc.annotations.Inject
import org.hibernate.Session

class Index
{
    @Inject
    Session session

    def getRecentPostings()
    {
        session.createQuery("from Posting order by posted desc").setMaxResults(20).list()        
    }
}

A few notes; the session gets injected even though the property is not private; Groovy is creating a private instance variable and a getter/setter for the property. That's just perfect.

I also like how succinct the getRecentPostings() method is. In previous apps, I tended to create a DAO service to hide some of the Java Generics ugliness, but that just stops being an issue here. With IDEA, I still get most of my editor support as well.

Tapestry has no knowledge of Groovy; IDEA is compiling the .groovy files to .class files and Tapestry is loading them. That's the beauty of the JVM.

For this simple example, we could write Java code that is nearly as succinct, but going forward I see no reasons why the Groovy code will not be significantly shorter, simpler and more readable than the equivalent Java code.

There's no reason to limit the use of Groovy to the pages and components. Entity classes in Groovy are also nice:

package com.nfjs.hls.blog.entities

import javax.persistence.*
import org.apache.tapestry5.beaneditor.DataType
import org.apache.tapestry5.beaneditor.NonVisual
import org.apache.tapestry5.beaneditor.ReorderProperties
import org.apache.tapestry5.beaneditor.Validate

@Entity
@ReorderProperties ("title,posted,content")
class Posting
{
    @Id
    @GeneratedValue (strategy = GenerationType.IDENTITY)
    @NonVisual
    Long id

    @ManyToOne (optional = false)
    Blog blog

    @Column (nullable = false)
    @Validate ("required")
    String title;

    @Column (nullable = false)
    @Validate ("required")
    @DataType ("longtext")
    String content

    @Column (nullable = false)
    Date posted
}

The @Validate annotations are Tapestry-specific and are being applied to the fields (allowing this is a new feature in 5.0.12). Because the getters and setters are generated without line numbers, the order of the properties ends up being a jumble; the @ReorderProperties annotation puts them into a reasonable order. You win some, you lose some.

On the services side of things, Groovy is a big win with Tapestry IOC. I have an AppModule to configure a few things, such as turning off production mode (which turns on full exception reports), and I want to override part of my Hibernate configuration in development mode:

package com.nfjs.hls.blog.services

import org.apache.tapestry5.SymbolConstants
import org.apache.tapestry5.hibernate.HibernateConfigurer
import org.apache.tapestry5.ioc.MappedConfiguration
import org.apache.tapestry5.ioc.OrderedConfiguration
import org.apache.tapestry5.ioc.annotations.Symbol

class AppModule
{

    def contributeApplicationDefaults(MappedConfiguration configuration)
    {
        configuration.add SymbolConstants.PRODUCTION_MODE, "false"
    }
  
    def contributeHibernateSessionSource(OrderedConfiguration configuration,

                                         @Symbol ("tapestry.production-mode")
                                         boolean productionMode)
    {
        if (!productionMode)
        {
            configuration.add "DevMode", {conf ->
                conf.setProperty "hibernate.show_sql", "true"
                conf.setProperty "hibernate.format_sql", "true"
                conf.setProperty "hibernate.hbm2ddl.auto", "create"
            } as HibernateConfigurer
        }
    }
}

The ability to wrap a closure as a simple interface (the kind of callback interface used throughout Tapestry) is very, very useful. The only fly in the ointment is GROOVY-2827, which keeps me from using SymbolConstants.PRODUCTION_MODE with the @Symbol annotation on the productionMode parameter.

I think I can get used to the Groovy syntax; I actually prefer the Groovy closure syntax to the Ruby block syntax though I generally prefer the Ruby approach elsewhere.

I expect to write all of this application in Groovy, and that may ultimately become an overall recommendation. Just us crazy framework authors should be coding in pure Java!

IntelliJ Diana Growls

That was a surprise: I downloaded IntelliJ 8 preview (Diana) and after it compiles, it uses Growl (the centralized Mac notification app) to inform me that its completed. Nice.

I'm also experimenting with using the Mac OS X look and feel (rather than Alloy). I had used Alloy in the past for performance reasons but IntelliJ 8 feels a bit faster. In addition, it's nice having the menu bar be where it's supposed to be ... the Mac look and feel puts it on the Mac menu bar, but Alloy puts it, Windows-style, in each top-level frame.

Nearly ready for 5.0.12

Tapestry 5.0.12 will be available pretty soon; I'm working on one very important component that's tricky to get Just Right: AjaxFormLoop. This is a component that allows you to dynamically add and remove rows (or divs, list items, whatever) that represent detail objects under a master object ... think line items in an invoice. The trick, of course, it to make it useful as-is, and to make it easy to customize it. It takes a bit of state-management gymnastics to make everything work just right, but it's almost there.

I expect to get this finished up in the next couple of days along with other work, and create a 5.0.12 release early next week.

Tapestry 5 with NetBeans

At NFJS Boston last month, I ran into Alex Kotchnev. We had a number of chats about Tapestry and spurring wide adoption. I'm still working on some of those ideas. He's a NetBeans user whereas most of the documentation assumes Eclipse or IDEA. He's posted a blog about use Tapestry in NetBeans; specifically, using the Maven support to avoid typing the dreaded Maven project creation incantation.

Tapestry for Nonbelievers at InfoQ

Renat and Igor's long awaited article, Tapestry for Nonbelievers is finally available. Check it out!

My only real criticism is that, had they focused on Hibernate, some of the code that switches back and forth between entities and entity ids would be handled automatically by tapestry-hibernate.

They're promising to follow through with more articles. I welcome it!

Improvements to the Tapestry 5 Tutorial

I spent some time yesterday revamping the Tapestry 5 Tutorial; you can see the updates at the nightly build site.

In short order we turned the Address object into a Hibernate entity, and stored it in a MySQL database, then used a Grid to show the added Addresses. Later improvements to the Tutorial will show editting and removal of Addresses.

I think the correct reaction to this would be "Dude, where's my code?". The application code for this app is so small you'd think something was missing. But that's what's Tapestry is all about ... providing the structure so that the framework can do the busy work.

I also updated some of the existing screen shots to show the "error bubbles" style of client-side validation that's been in T5 for quite some time.

People have been pining for a 5.0.12 release, but I'm glad I've been holding it back; I've been having a chance to fix bugs, and finding many annoyances as I work on a client project. Working in earnest on a project is always the best way to find the rough edges, and the end result is much more polished.

I think I must be a harsher critic of Tapestry than most; I frequently add bugs along the lines of "when I screw up, Tapestry should tell me how to fix it".

Tapestry: Components + Aspects

When I was first starting on the Tapestry 5 code base, I started using AspectJ for parts of the framework. I wrote some clever aspects for defensive programming (cementing "don't accept null, don't return null" as an iron-clad rule) and created some aspects to simplify concurrent access to certain resources.

I certainly never expected Tapestry developers to have to use AspectJ: it's certainly a powerful tool, but it has downsides: AspectJ affects the build process in a signifcant way, it's tricky to use properly, it adds a huge runtime dependency and the plugin for Eclipse made the IDE slow(-er) and (more) unstable. I eventually stripped out the AspectJ code entirely and restricted the framework to traditional code (plus, of course, Javassist).

In the interrum a number of aspect oriented touches have appeared in Tapestry. Certainly, the entire mixins concept is an approach to aspect orientation. Likewise, service decorators. In fact, much of the class transformation mechanism is a way of performing aspect-oriented operations on Tapestry component classes. Creating your own parameter binding prefixes (something I often do for client projects) is another aspect: one focused on streamlining how data gets into or out of a component.

However, all of these techniques can be cumbersome; they are built on the Javassist "language" which is another tool that can be difficult to use properly. I've created layers around Javassist to make things easier, but it's still an uphill battle.

Earlier in the week I was able to take the Javassist out of service method advice. However, that's not quite enough ... I want it to be easier to apply advice to component methods just as easily.

Well, I did just that. There is a ComponentMethodAdvice that can be applied to component methods. It's only a slight variation from the MethodAdvice used on service interfaces (the difference being that the invocation allows access to the ComponentResources of the component).

Here's a concrete example: Part of tapestry-hibernate is the @CommitAfter annotation, which is used to commit the running transaction after invoking a method. There's a decorator that can be applied to services, and there's a worker that is automatically threaded into the Component Class Transformation chain of command.

The worker used to use the Javassist approach to decorate method invocations with the desired logic: commit the transaction when the method completes normally, or with a checked exception. Abort the transaction when the method throws a runtime exception.

First, the old code:

public class CommitAfterWorker implements ComponentClassTransformWorker
{
    private final HibernateSessionManager _manager;

    public CommitAfterWorker(HibernateSessionManager manager)
    {
        _manager = manager;
    }

    public void transform(ClassTransformation transformation, MutableComponentModel model)
    {
        for (TransformMethodSignature sig : transformation.findMethodsWithAnnotation(CommitAfter.class))
        {
            addCommitAbortLogic(sig, transformation);
        }
    }

    private void addCommitAbortLogic(TransformMethodSignature method, ClassTransformation transformation)
    {
        String managerField = transformation.addInjectedField(HibernateSessionManager.class, "manager", _manager);

        // Handle the normal case, a succesful method invocation.

        transformation.extendExistingMethod(method, String.format("%s.commit();", managerField));

        // Now, abort on any RuntimeException

        BodyBuilder builder = new BodyBuilder().begin().addln("%s.abort();", managerField);

        builder.addln("throw $e;").end();

        transformation.addCatch(method, RuntimeException.class.getName(), builder.toString());

        // Now, a commit for each thrown exception

        builder.clear();
        builder.begin().addln("%s.commit();", managerField).addln("throw $e;").end();

        String body = builder.toString();

        for (String name : method.getExceptionTypes())
        {
            transformation.addCatch(method, name, body);
        }
    }
}

Can you tell where that logic (when to commit and abort) is? Didn't think so, it's obscured by lots of nuts-and-bolt logic for creating the Javassist method body.

All that stuff with the BodyBuilder is a way to assemble the Javascript method body piecewise. It's certainly meta-programming, but it's only a few steps ahead of writing the Java code to a temporary file and compiling it. That's great for people into the meta-coding concept (count me in) but sets the bar so high that most people will never be able to tap into the power.

That's where the advice comes in: the new version of the worker is much simpler:

public class CommitAfterWorker implements ComponentClassTransformWorker
{
    private final HibernateSessionManager _manager;

    private final ComponentMethodAdvice _advice = new ComponentMethodAdvice()
    {
        public void advise(ComponentMethodInvocation invocation)
        {
            try
            {
                invocation.proceed();

                // Success or checked exception:

                _manager.commit();
            }
            catch (RuntimeException ex)
            {
                _manager.abort();

                throw ex;
            }
        }
    };

    public CommitAfterWorker(HibernateSessionManager manager)
    {
        _manager = manager;
    }

    public void transform(ClassTransformation transformation, MutableComponentModel model)
    {
        for (TransformMethodSignature sig : transformation.findMethodsWithAnnotation(CommitAfter.class))
        {
            transformation.advise(sig, _advice);
        }
    }
}

Really simple! The advice is very clear on what happens when, as long as you realize that proceed() invokes the original method (the method "being advised"), and also Tapestry's way of handling thrown exceptions: runtime exceptions are not caught by proceed(), but checked exceptions are.

This simplicity opens up the power of aspects without all mental overhead of AspectJ's join points and declarative point-cut language. It's only a fraction of AspectJ's power, but it's a very useful fraction.

Further, AspectJ works entirely in a static way (it primarily works at build time), but this style of AOP is a mix of declarative and imperative. In other words, we can have active code decide on a component-by-component and method-by-method basis what methods should be advised. The above example is typical: it's driven by an annotation on the method ... but other cases jump to mind: triggered by naming conventions, by parameter or return types, or by thrown exceptions. And there's no guesswork about going from the advice to Tapestry services: its the same uniform style of injection used throughout Tapestry.

In the future, we may add other options, such as advice for field access, but what's just checked in is already a huge start.

Tapestry AOP: MethodAdvice

I don't go out of my way to talk about Tapestry 5 IoC, since it is really just a side issue compared to the main framework (but it is the infrastructure that allows Tapestry to be everything it is).

However, the IoC container is quite powerful and fully featured and quite useful on its own. However, one feature of the container wasn't quite up to snuff: service decorators. Service decorators are a limited form of aspect oriented programming wherein its possible to "wrap" a service with an interceptor that provides additional behavior. Tapestry includes two decorators: one for adding logging of method invocations, results and exceptions and a second decorator for managing Hibernate transactions.

Tapestry is responsible for invoking a decorator method, but the responsibility of that method was to return an interceptor object that encapsulates the behavior. So, a logging interceptor is responsible for logging method entry and exits around re-invoking each method on the service, and a transaction interceptor is responsible for commiting or aborting the transaction after invoking the service method.

The problem is how to provide such an interceptor. Option A is to use a JDK dynamic proxy. This is a perfectly good solution, but has a few annoyances; you drop out of pure bytecode which means that Hotspot will not be able to fully optimize your code. In addition, there a minute cost associated with constantly boxing and unboxing primitives and packaging parameters up as an object array.

Option B is to use Tapestry's ClassFactory and ClassFab interfaces to create a new class and then instantiate. Javassist is under the surface, but the APIs are tamed. Even so, its a lot of work and a lot to test to check all the edge cases such as handling of primitives (more boxing and unboxing). This was the approach I've been taking in the past.

So I've taken some inspiration from the AOP Alliance approach.

You can now create an interceptor using a special service, AspectDecorator, and providing MethodAdvice. For example, the HibernateTransactionDecorator (which looks for the @CommitAfter annotation) is simplified to:

public class HibernateTransactionDecoratorImpl implements HibernateTransactionDecorator
{
    private final AspectDecorator _aspectDecorator;

    private final HibernateSessionManager _manager;

    private final MethodAdvice _advice = new MethodAdvice()
    {
        public void advise(Invocation invocation)
        {
            try
            {
                invocation.proceed();
            }
            catch (RuntimeException ex)
            {
                _manager.abort();

                throw ex;
            }

            // For success or checked exception, commit the transaction.

            _manager.commit();
        }
    };

    public HibernateTransactionDecoratorImpl(AspectDecorator aspectDecorator, HibernateSessionManager manager)
    {
        _aspectDecorator = aspectDecorator;

        _manager = manager;
    }

    public <T> T build(Class<T> serviceInterface, T delegate, String serviceId)
    {
        String description = String.format("<Hibernate Transaction interceptor for %s(%s)>",
                                           serviceId,
                                           serviceInterface.getName());

        AspectInterceptorBuilder<T> builder = _aspectDecorator.createBuilder(serviceInterface, delegate, description);

        for (Method m : serviceInterface.getMethods())
        {
            if (m.getAnnotation(CommitAfter.class) != null)
            {
                builder.adviseMethod(m, _advice);
            }
        }

        return builder.build();
    }
}

At this weekend's No Fluff Just Stuff, Neal Ford went out of his way to point out how hard meta-programming in Java is, and how impressed he was by what I've accomplished using meta-programming throughout Tapestry. Even so, he and the other speakers voted me "most in need of a dynamic programming language" behind my back!

Meanwhile, this MethodAdvice approach (the terminology is borrowed from AOP and AspectJ) really is a very succinct way of expressing this transaction concern, and it's so easy to mix and match methods within the interface, advising some of them and letting the others just "pass thru". There's going to be a lot of room to use this same technique to provide many other cross-cutting concerns such as performance monitoring, security checks ... even checking for null pointers!

Service Dependencies in Tapestry 5

I was curious to see if I could generate a diagram that showed how all the services inside Tapestry 5 are interconnected. In the diagram, solid lines are dependencies, dashed lines are contributions, and dotted lines represent a services that listens to events from another service.

The final diagram is a bit complex. Ok, that's a tremendous understatement. Yellow nodes are public services, grey nodes are internal services, and orange nodes are simple beans (contributed into services) ... those beans, though, may still have their own dependencies. I think OmniGraffle choked on the complexity (why do some of the lines zip around so aimlessly?) and kind of "phoned it in" in terms of layout. I had hoped for something that could go on a T-shirt.

Worse yet, this is only a partial diagram; I didn't cover many of the dependencies that arrive as contributions to services (there's just a few of those in there). Those contributions would show up as even more orange nodes; these additional nodes would provide connections to many of the outliers that appear to be unreferenced.

Is this good or bad? Is it too complex? Can it be simplified? Does all that really need to be in there?

I think it is good: When I'm coding, I find it easy enough to work from a service interface, to the implementation of the interface, to the dependencies of the implementation. This works for me when developing Tapestry itself, or writing apps in Tapestry. Since there are so many small objects, no single object is large, or complex, or hard to understand or debug. It's a bit daunting to know where to start, I'm sure. However, only a few of these services are all that relevant for day-to-day development of Tapestry.

The moral of the story is that simplicity is hard! In Tapestry, pages and templates are as simple and straight forward as I think is possible. There's no excess, no distractions, no clutter, no boilerplate. Much of the logic represented in the diagram is to "fill around the edges" of this simplicity, to bridge the gap from the simple Tapestry component model to the much more complex world of stateless HTTP and multi-threaded request handling.

New Tapestry 5 Features

I've been using Tapestry 5 for a client project, which is a great chance to find the rough edges. If you've been following the bug list, you'll have seen lots of minor bug fixes, many related to JavaScript.

I've also added a couple of powerful new features.

The Grid component now has an inplace parameter; setting this to true "Ajax-ifies" the Grid; it updates in place. Clicking any of the paging or sorting controls will repaint the Grid in place, without affecting the rest of the page. Handy. I had originally though in terms of creating an AjaxGrid component for this purpose capable of redrawing individual cells ... now I'm not so sure that's necessary.

Perhaps more importantly, I've added a new annotation, @CommitAfter, that can be placed on component methods, to indicate the Hibernate transaction should be committed after invoking the method. It can also be used with services, via a decorator. In any case, much of the work was contributed by Igor Drobiazko.

These features are available in the nightly build, and will be part of Tapestry 5.0.12.

They are also a good indication of how Tapestry future backwards compatibility will work. The new behaviors "slots in" with the existing behavior. In this case, a new annotation triggers new behavior, or a new parameter is added to an existing component. These are the vectors by which Tapestry adapts to your code (and not vice-versa).

In the future, we'll be adding much more dramatic new features, as well as integrations with other frameworks, such as Seam, Spring Web Flow, Lucene, Quartz and other parts of the evolving standard open source stack. But due to the baked in elegance of Tapestry, they will integrate with barely a ripple.

A round-up of Tapestry blogs

I was just fiddling with Google's Blog Search and searched for "java tapestry"; here's a few things I stumbled upon:

• Someone named "Audible Smirk" is running their own NCAA-style Java web framework tournament
• Alex Kotchnev is looking to create Grapestry: a Tapestry front end with a Grails back end (but seems to be using Tapestry 4)
• Tom Davies is Combining Tapestry 5 with the functional programming language CAL
• I found an excerpt from Alexander's book that covers building advanced components

Fun stuff ... and I have more advancements for Tapestry 5 about ready to announce!

Web Framework Smackdown: Tapestry

I wasn't able to attend this years TSSJS and I really would have liked to be in on Matt Raible's web framework smackdown. Matt has raised a number of questions, here are the responses for Tapestry 5:

• What is the overall performance of your framework as it compares to others?

  I haven't done any direct performance comparisons, but Tapestry 5 is considerably faster than Tapestry 4, which was already "fast enough". Using FireBug, I see that most requests (on my development machine) are on the order of 4 - 6 ms to process a request and get the response stream back; it takes the browser much longer to render the HTML up onto the screen. Tapestry uses no runtime reflection and doesn't appear to have any concurrency bottlenecks (the way OGNL can be in Tapestry 4).

• How does your web framework position themselves in relation to Web Beans?

  Web Beans is one way of looking at how to manage state and process in a web application, but not the only one. Tapestry is already being used with Seam, and integration with Spring Web Flow will come some day (likely 5.1 as we're trying to get a stable 5.0 release out the door). Tapestry is in a good position to "embrace and extend" these kinds of specifications, since it doesn't care about anything but JavaBeans properties.

• How easy is it to create a re-useable component in your framework? Is it as easy as sub-classing an existing component?

  You can subclass an existing component, but you can create a component as just a POJO with no inheritance. There's no configuration (other than putting it in the correct components package for your application). Bi-directional component parameters are as easy as placing a @Parameter annotation on a field. Components may have templates and may contain other components, it's all very consistent. It's always been easy to create components in Tapestry, but Tapestry 5 adds a number of features that make it even easier (such as no longer requiring you to select the correct base class). Further, it is much easier in Tapestry 5 to subclass a component and inherit its template and message catalog, but add new functionality. In addition, Tapestry 5 has the concept of Mixins, which provide another way to add new functionality to an existing component.

• What is the key differentiating characteristic of your framework that makes it better than the rest?

  A ruthless emphasis on developer productivity drives nearly all the design decisions in Tapestry: this shows up with live class reloading, exceptional error reporting, adaptive APIs, and minimizing the amount of code (and markup) in an application. In terms of implementation, Tapestry embraces pervasive object orientation (not just inheritance), meaning lots of small focused objects able to work together seamlessly.

• What do you think about the various scopes introduced by Seam, e.g. conversation vs request or session? If you support these additional scopes, do you also provide some sort of concurrency control?

  These are very important; Tapestry already has a number of ways of managing server-side state; individual page or component fields may be stored on the client, rather than in the session, for example. As with much of T5, the support is cleanly extensible, so natively supporting conversational state without using Seam is a possibility, as is leverage Seam itself (there's an add-on library for this), as are many other options.

• Why can't we, the Java Community, come together and adopt the best application framework and settle the web development subject?

  That's like saying, "Why can't we, the developer community, come together and adopt the best language?". Generally, people with a lot of experience in one framework are looking for a bigger, badder version of their framework, that also magically solves all their problems ... especially those caused by their choice of framework. Sometimes it's the craptaculous framework. Often the fix is something is a completely different vein (and Tapestry has always been that different vein). It's kind of like the adage about a council of Great Apes designing their evolutionary successor. They'd probably come up with a bigger, stronger Ape ... not a scrawny, weak, hairless human.

• What are you doing to help with developer productivity?

  Tapestry is primarily focused on developer productivity, and this should not be confused with simply adding tool support (and dealing with wizard-generated code). Tapestry addresses the needs of developers to build applications using simple, off the shelf tools, such as Java IDEs and standard XML editors. Tapestry has best-of-breed exception reporting. This is more than a fancy exception report page; it's also attention to detail throughout the framework to catch problems and report them in a useful manner to the developer. For example, if you use the wrong name when defining a component or referencing a page, Tapestry doesn't just throw a NullPointerException the way too many libraries would; it clearly identifies the line in your template with the problem, identifies the name you gave, and provides a list of possible values to help you make the correction. Oh, and it's generally case insensitive. Add live class reloading on top of this, and you can burn through your application very quickly. In fact, Tapestry addresses developer productivity by making the development more Fun and less of a Drudge.

• 2008 is a huge year for the mobile web. How do you help developers build great mobile web applications?

  Size counts on the mobile web and Tapestry has been built with terseness as a virtue. URLs are shorter and more readable, page content is simpler, and many components include a "lean" mode to reduce optional content generation (such as CSS classes inside the Grid component).

• If you couldn't use your framework, what would you use and why?

  I'd be very interested in learning to really leverage Adobe's Flex ... if I can't use Tapestry, I want something that gives me pixel-level control and good abstraction without the kludges and half measures of JavaScript and the DOM.

• How do you enable rich Ajax applications?

  Tapestry supports very clean interactions between the client and the server, as server-side components can easily be referenced via a URL. Handling Ajax requests in Tapestry is virtually identical to handling of traditional page-oriented requests. A limited amount of base support is provided in the framework (text field autocompletion, dynamic content zones, extensible forms) with much more support provided by the community. Currently Ajax support is a wrapper on top of Prototype/Scriptaculous and there is great desire to make the client-side support more agnostic.

• Can a developer make a change to source, and hit RELOAD in the browser to see the change? If not, why not?

  Absolutely. I think Tapestry was on the cutting edge nearly two years ago when this feature was introduced in early versions of Tapestry 5; other frameworks are still trying to catch up. This feature is critical to developer productivity and making it fun. Further, Tapestry's approach to server-side state means that persistent data is not lost on reload, which significantly enhances the developer's ability to continue work from where they left off. Tapestry is even pretty good about picking up the addition of new pages and components, not just updates to existing Java classes.

• What do you think about the whole Flex revolution, and do you think you are competitors to this technology?

  Flex is an impressive piece of work, and it gives you quite a lot of power. Still, I think RIAs (such as Flex) are in a parallel, not competing, category against traditional web applications (even those enhanced with Ajax) . Tapestry has an emphasis on bookmarkable URLs, and though Flex supports this feature, there are some costs and limitations. I don't want to spread FUD, but to me a Flex application is pretty heavy weight ... big enough to require a loading screen, whereas a web application just pops up instantly. I haven't pushed myself far enough up the Flex learning curve to talk in more detail. I tend to discount the "Grandma factor": that is, Flash is almost certainly already installed on Grandma's computer. Even so, Flex's greatest strength and greatest weakness is the proprietary Flash runtime: even more than Sun, Adobe is protective of their runtime environment, and fearful of splintering. This is increasingly unsettling to developers used to full open source everything.

  P.S. I'm sure James Ward will correct everything I've just stated.

• How easy is it to create a module and plug it into a bigger application, complete with configuration, code, and view?

  Really, really easy. Choose a package name, put your pages, components, base classes, etc. in sub-packages, along with any templates. Create a module class for your library to provide Tapestry with configuration for the module's package. One tweak to your Ant or Maven build script to enable autoloading of the module, at which point it Just Works. Tapestry takes care of all sorts of issues, including providing secure and efficient client-side access to assets (images, stylesheets, etc.) packaged inside your library's JAR.