Thursday, June 26, 2008

IntelliJ: Flip Equals

Just hit a NullPointerException in some code:

    public boolean isOwner()
    {
        return authManager.getUser().equals(blog.getOwner());
    }

Turns out, sometimes getUser() returns null. I started to retype this, then thought: "Can IntelliJ help me?"

Answer: yes. Because of IntelliJ coolness, I click anywhere in the expression, type option-enter and choose 'Flip .equals()' and it rewrites the code to:

    public boolean isOwner()
    {
        return blog.getOwner().equals(authManager.getUser());
    }

Twitku

I just noticed that Ted Neward is on Twitter ... but only twits in Haiku form.

Monday, June 23, 2008

Maven for dependencies, not building

I've been working a bit with Ivy but finding it doesn't quite fit my needs (or that I have a lot more to learn). What I've found with Ivy is that it has its way of doing things, and part of The One True Path is that you should create a repository just for your organization ... they really hate the idea of depending on the central Maven repository.

In any case, my need is to make it easy to build Tapestry or Tapestry demos from source without a lot of fuss. That's where I like Maven's dependency management, especially for the most common artifacts.

The other problem I've had with Ant is that it doesn't get Maven scopes perfectly, or Maven artifacts (such as source JARs) without a lot of redundant typing.

So, for the mean time, I'm back to Ant plus the Maven Ant tasks.

What I want is to create and manage four folders:

  • runtime
  • runtime-src
  • test
  • test-src

The main folders, runtime and test, contain JARs needed for compilation/execution of the production code, and for the test code. This could expand to include more of Maven's scopes in the future, such as provided, but it suits my current needs. The -src folders contain source JARs, which I find essential to being productive. Nothing irks me more than opening a base class or interface and getting that ugly view of just the method names because source isn't available. It gives me flashbacks to miserable sessions with WebLogic trying to figure out what the hell it was doing.

So, how can we get there without the rest of Maven? How about this:

<?xml version="1.0"?>
<project name="common" xmlns:mvn="urn:maven-artifact-ant">

    <!-- Names of common directories used in the build. -->

    <!-- Directory in which temporary files are created. -->
    <property name="target.dir" value="target"/>
    <property name="target.lib.dir" value="target/lib"/>

    <!-- Directory to which imported dependencies are placed. Four subfolders are created: runtime, runtime-src, test
         and test-src. -->
    <property name="lib.dir" value="lib"/>

    <path id="maven-ant-tasks.classpath" path="maven-ant-tasks-2.0.9.jar"/>

    <typedef resource="org/apache/maven/artifact/ant/antlib.xml" uri="urn:maven-artifact-ant"
             classpathref="maven-ant-tasks.classpath"/>


    <macrodef name="import-dependencies">
        <attribute name="scope"/>
        <attribute name="folder" default="@{scope}"/>
        <element name="dependencies" implicit="true"/>

        <!-- locals -->
        <attribute name="lib" default="${lib.dir}/@{folder}"/>
        <attribute name="lib-src" default="@{lib}-src"/>

        <attribute name="target-lib" default="${target.lib.dir}/@{folder}"/>
        <attribute name="target-lib-src" default="${target.lib.dir}/@{folder}-src"/>

        <attribute name="lib-fileset" default="@{scope}.dependency.fileset"/>
        <attribute name="src-fileset" default="@{scope}.dependency.sources.fileset"/>

        <sequential>
            <mvn:dependencies pomrefid="project.pom" verbose="true" usescope="@{scope}"
                              filesetid="@{lib-fileset}"
                              sourcesfilesetid="@{src-fileset}">
                <remoteRepository id="tapestry-nightly"
                                  url="http://tapestry.formos.com/maven-snapshot-repository">
                    <snapshots enabled="true"/>
                </remoteRepository>
                <remoteRepository id="maven-central"
                                  url=" http://repo1.maven.org/maven2"/>
                <remoterepository id="openqa-release"
                                  url="http://archiva.openqa.org/repository/releases/"/>
            </mvn:dependencies>

            <mkdir dir="@{target-lib}"/>
            <mkdir dir="@{target-lib-src}"/>
            <!-- Flatten them, so we can do a sync. -->
            <copy todir="@{target-lib}" flatten="true" preservelastmodified="true">
                <fileset refid="@{lib-fileset}"/>
            </copy>

            <copy todir="@{target-lib-src}" flatten="true" preservelastmodified="true">
                <fileset refid="@{src-fileset}"/>
            </copy>


        </sequential>
    </macrodef>

    <macrodef name="remove-overlap">
        <attribute name="source"/>
        <attribute name="target"/>

        <attribute name="fileset.property" default="@{source}.overlap.files"/>

        <sequential>
            <pathconvert pathsep="," property="@{fileset.property}">
                <fileset dir="@{source}"/>
                <flattenmapper/>
            </pathconvert>

            <delete dir="@{target}" includes="${@{fileset.property}}"/>
        </sequential>
    </macrodef>

    <target name="setup-libs" description="Copy dependencies to lib folder.">

        <mvn:pom file="pom.xml" id="project.pom"/>

        <!-- Delete the scratchpad space. -->
        <delete dir="${target.lib.dir}" quiet="true"/>

        <import-dependencies scope="runtime"/>

        <!-- For the moment, this is somewhat broken: test scope is a super-set of compile/runtime scope.
             Everything in the runtime folders will end up in the test folders, plus more. -->
        <import-dependencies scope="test"/>


        <!-- Snapshots come down with the datestamp/version number, we need to fix that. -->
        <move todir="${target.lib.dir}" preservelastmodified="true">
            <fileset dir="${target.lib.dir}"/>
            <!-- Turn the date/version stamp back into SNAPSHOT -->
            <regexpmapper from="^(.*)(\d{8}\.\d{6}\-\d+)(.*)$$" to="\1SNAPSHOT\3"/>
        </move>

        <!-- Delete the overlap between lib/runtime and lib/test -->

        <remove-overlap source="${target.lib.dir}/runtime" target="${target.lib.dir}/test"/>
        <remove-overlap source="${target.lib.dir}/runtime-src" target="${target.lib.dir}/test-src"/>

        <echo>*** Synchronizing ${lib.dir} ...</echo>
        <sync todir="${lib.dir}" verbose="true">
            <fileset dir="${target.lib.dir}"/>
        </sync>
        <echo>... done.</echo>

        <delete dir="${target.lib.dir}" quiet="true"/>
    </target>

</project>

Just add the Maven Ant tasks JAR and a pom.xml (that exists exclusively to identify dependencies) and you are off and running.

Some of my requirements are awkward to implement in Ant:

  • Flattened directories; Maven really wants to build a mirror of the repository, with directories for groups, but I want everything in a single directory.
  • Remove overlap; Maven's test scope includes the runtime scope, extra work is necessary to keep the test directory to just the additional JARs without redundantly including what's already in runtime
  • Timestamps; I want to sync the directories, not just copy in, to account for version number changes and I don't want to change timestamps unless a file has changed (to keep the IDE from wasting time rescanning and reparsing it)

The script creates a temporary directory under target, which is first used to flatten the copy from the local repository, then to remove the runtime vs. test redundancies before sync-ing it over the proper lib directory.

The best part of all this? Nothing happens until ant setup-libs rather than the frequent surprises that Maven gives you!

Eventually, I'll repackage some of this into a common build.xml that can be shared across modules. In the meantime, it's good enough for demos and labs in the workshop.

Thursday, June 19, 2008

Firefox 3 : The Good, the Bad

The Good is that it does feel faster,and it now sports a UI that looks much more Mac native.

The Bad is that many plugins don't work yet, such as FireBug! In addition, the Selenium support for Firefox is glitchy with the new release.

Update: People are coming out of the woodwork to point me at the Firebug for FF3 ... check the comments!

Wednesday, June 11, 2008

Groovy: Leaky Abstractions and Confusing Exceptions

Here's an odd place to get a ClassCastException:

  • org.hibernate.type.StringType.toString(StringType.java:44)
  • org.hibernate.type.NullableType.nullSafeToString(NullableType.java:93)
  • org.hibernate.type.NullableType.nullSafeSet(NullableType.java:140)
  • org.hibernate.type.NullableType.nullSafeSet(NullableType.java:116)
  • org.hibernate.param.NamedParameterSpecification.bind(NamedParameterSpecification.java:38)
  • org.hibernate.loader.hql.QueryLoader.bindParameterValues(QueryLoader.java:488)
  • org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1554)
  • org.hibernate.loader.Loader.doQuery(Loader.java:661)
  • org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
  • org.hibernate.loader.Loader.doList(Loader.java:2211)
  • org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2095)
  • org.hibernate.loader.Loader.list(Loader.java:2090)
  • org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:375)
  • org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:338)
  • org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:172)
  • org.hibernate.impl.SessionImpl.list(SessionImpl.java:1121)
  • org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
  • org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:811)
  • sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  • sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  • sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  • java.lang.reflect.Method.invoke(Method.java:585)
  • org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
  • groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
  • groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
  • groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
  • org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
  • org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
  • org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
  • org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethod0(ScriptBytecodeAdapter.java:198)
  • com.nfjs.hls.blog.services.AuthenticationManagerImpl.authenticate(AuthenticationManagerImpl.groovy:34)

That StringType object was expecting a string, but got a org.codehaus.groovy.runtime.DefaultGroovyMethods$2. Seems like a little bit of Groovy has leaked unexpectedly into my Java.

The context is a method for authenticating a user's handle and password:

   Blogger authenticate(String handle, String plaintextPassword)
    {
        def authenticationCode = computeAuthenticationCode(handle, plaintextPassword) 

        def query = session.createQuery("from Blogger where handle = :handle and authenticationCode = :code")

        query.setProperties([handle: handle, code: authenticationCode])

        def result = query.uniqueResult()

        if (result)
            asm.get(UserCredentials.class).userId = result.id;

        return result
    }

   def computeAuthenticationCode(handle, plaintextPassword)
    {
        def md = MessageDigest.getInstance("MD5")

        // Salt the digest with the lower-cased handle
        md.update(handle.toLowerCase().bytes)
        md.update(plaintextPassword.bytes)

        md.digest().encodeBase64()
    }

A little hunting around showed that the ClassCastException was related to the authentication code.

Now the encodeBase64() method should be returning a String, one would think. Ah, a little poking around in the Groovy JDK docs and in fact, it returns Writable. After a few experiments, I found that the following change worked and was most pleasing:

   md.digest().encodeBase64() as String

This is the exact kind of error that static typing is designed to catch. Certainly, this would be caught by some tests eventually (I'm working a little fast and loose here), but in the end I had to work with the debugger to chase down what was wrong.

And that brings up a more significant problem: it appears that in Groovy, local variables are not visible in the debugger. I suspect they may be stuck inside the metaClass object or somewhere else I haven't found yet. That is significantly less than ideal. Update: User error; there was a button in the debugger pain that made the local variables visible.

What I'm finding, working with Groovy, is that for the trivial kind of code I'm writing to support my application, any extra bang I get for using Groovy instead of Java is being offset by leaky abstractions, my own lack of familiarity with Groovy, and the very shakey developer tools (even inside IDEA!). One aspect of this is just how trivial the Java code for a Tapestry application is; there's just not much room for Groovy to improve on pure and simple Tapestry Java.

However, the exercise is certainly worth it, because many people will be using Groovy with Tapestry and it should work without any hickups.

Tuesday, June 10, 2008

JavaScript var vs. Groovy def

I keep typing the following:

var blogger = new Blogger()
when I should be saying
def blogger = new Blogger()

def is used to define new methods and variables. The problem is that the JavaScript syntax compiles ... it just doesn't execute. I looks like a call to method var().

The code fails at runtime with groovy.lang.MissingPropertyException: "No such property: blogger for class: com.nfjs.hls.blog.pages.Login".

Just takes a little getting used to. Perhaps this is where polyglot programming breaks down a little ... switching between too similar languages.

Monday, June 09, 2008

Using Groovy for IoC Service Decorators

Continuing with my experiments with Groovy, I took a pass at writing a service decorator in Groovy. In Tapestry, the RequestHandler service is the primary execution path; everything filters through it. It's an extensible service, based on the pipeline pattern, so I could contribute a filter into the service, but instead I wrote a decorator around the service.

    def decorateRequestHandler(service, Logger logger)
    {
        {request, response ->
            def start = System.nanoTime()

            def result = service.service(request, response)

            def elapsed = System.nanoTime() - start

            logger.info(String.format("Request time: %5.3f s -- %s", elapsed * 10E-9d, request.path))

            return result
        } as RequestHandler
    }

Tapestry's naming convention is that a module class method named decorateXXX is a decorator for service XXX. The service implementation is passed in. The return value must be the same type as the service (that is, implement the service's interface). Again, we use a closure in Groovy as an easy alternative to an inner class in Java. The code here logs the elapsed time (in seconds) for the request.

What's interesting is just how much gets into the stack trace:

 at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106)
 at $RequestHandler_11a6ead473a.service($RequestHandler_11a6ead473a.java)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:912)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
 at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:766)
 at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:754)
 at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
 at com.nfjs.hls.blog.services.AppModule$_decorateRequestHandler_closure2.doCall(AppModule.groovy:49)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
 at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:248)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:756)
 at groovy.lang.Closure.call(Closure.java:292)
 at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:48)
 at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:72)
 at $Proxy147.service(Unknown Source)
 at $RequestHandler_11a6ead4732.service($RequestHandler_11a6ead4732.java)

In this view, $RequestHandler_11a6ead4732 is a Tapestry runtime fabricated class, the proxy for the service. $Proxy147 is a JDK dynamic proxy which is eventually hooked up to the closure code. The service parameter (to the decorateRequestHandler) is the instance of CheckForUpdatesFilter.

What looks like a simple method invocation turns into several levels of indirection through the Closure and MetaClass code, and then ultimately a reflective method invocation before it picks back up with the first filter in the pipeline (CheckForUpdatesFilter).

This is not especially troubling; I suspect it would be all but impossible to measure the "extra" time the Groovy code takes (vs. an equivalent Java implementation). Still, this could easily add up inside a tight loop, such as the rendering state machine built into Tapestry components. Certainly Merlyn was having performance issues with his visual processing application.

Sunday, June 08, 2008

Real World Haskell

When I'm in the mood to feel really dense, I try to wrap my brain around functional programming. My poison of choice is Haskell, since that's the purest and least compromising of the languages, in my uninformed opinion. I have a couple of books on the subject and every once and a while I struggle with the syntax and the alternate mode of thought.

I recently stumbled across a very approachable on-line book-in-progress about Haskell: Real World Haskell.

I still have my doubts about functional programming; I should love it, because I tend to think about code and how code fits together all the time. However, the heavy mathematical bent of functional programming is a bit of a challenge, and the FP-ers seem to value conciseness over all other things. I worry that Haskell programs are write-only in the way that Perl can be; that every program is an elegant puzzle-box, but a puzzle nonetheless.

I'm still intrigued not due to the conciseness, or the promise of parallel execution, but because of lazyness. Having lazy evaluation built right into the language means that a lot of common coding conventions and patterns are simply a natural byproduct of the use of the language.

For example, throughout Tapsestry there are examples of code that is based on a recursive algorithm implemented non-recursively (typically, using queues). In Haskell, you get that for free pretty much universally: the Haskell execution stack does not get very deep, though there's a kind of stack of as-yet-unevaluated expressions (it looks like the Haskell folk call these "thunks") behind the scenes, acting like the queues I'm coding directly.

Because it's baked into the language, you are free to express algorithms very purely, without the implementations being obscured by performance details (such as converting recursion into queues). In fact, the laziness of Haskell is really good at handling infinite lists (as long as you don't ask for the length of that list!)

There's a bit of hype going on in the FP/Haskell/Erlang space. For example, people keep talking a lot about the need for FP to tackle large numbers of cores. Does the Emperor have any clothes here? I'm not certain: Haskell and Erlang use non-native threads. That's a great way to maximize a single core and all the no-sideffects, no-mutable state simplifies the problem vastly from a typical Java application.

However, Erlang typically approaches the highly scalable space by starting (and, as necessary, restarting) lots of processes. The immutable state must be extracted from one process and serialized to another; Erlang has a native format for this that's pretty efficient. Haskell adds similar functionality as an add-on library. But from my point of view, it doesn't feel that different from JMS and messaging with a bit of coding discipline.

In fact, the point of view of 1060 NetKernel is that you can have the advantages of an Erlang style model, but implement it in your choice of languages (Java, JavaScript, Python, etc., etc.). We may have a gig at Formos where I'll have a chance to really look into NetKernel, I've been hovering around the edges on it for a couple of years.

Tapestry with Groovy

While I'm waiting for the vote on Tapestry 5.0.12, I'm starting some work on a more complete demo application. As usual, I'm trying to teach myself a bunch of new things, such as using Ant and Ivy (in place of Maven), and using Groovy for my pages, entities and services.

I'm still having trouble with live class reloading in Tomcat, so I'm staying with Jetty for a bit longer.

The Groovy stuff for pages and such is just fine. My demo application is a simple blogging site (boring stuff, but a domain model everyone understands). I don't have much yet, but my first page lists recents blog postings:

<t:grid source="recentPostings"/>

And the page class is written in Groovy:

package com.nfjs.hls.blog.pages

import org.apache.tapestry5.ioc.annotations.Inject
import org.hibernate.Session

class Index
{
    @Inject
    Session session

    def getRecentPostings()
    {
        session.createQuery("from Posting order by posted desc").setMaxResults(20).list()        
    }
}

A few notes; the session gets injected even though the property is not private; Groovy is creating a private instance variable and a getter/setter for the property. That's just perfect.

I also like how succinct the getRecentPostings() method is. In previous apps, I tended to create a DAO service to hide some of the Java Generics ugliness, but that just stops being an issue here. With IDEA, I still get most of my editor support as well.

Tapestry has no knowledge of Groovy; IDEA is compiling the .groovy files to .class files and Tapestry is loading them. That's the beauty of the JVM.

For this simple example, we could write Java code that is nearly as succinct, but going forward I see no reasons why the Groovy code will not be significantly shorter, simpler and more readable than the equivalent Java code.

There's no reason to limit the use of Groovy to the pages and components. Entity classes in Groovy are also nice:

package com.nfjs.hls.blog.entities

import javax.persistence.*
import org.apache.tapestry5.beaneditor.DataType
import org.apache.tapestry5.beaneditor.NonVisual
import org.apache.tapestry5.beaneditor.ReorderProperties
import org.apache.tapestry5.beaneditor.Validate

@Entity
@ReorderProperties ("title,posted,content")
class Posting
{
    @Id
    @GeneratedValue (strategy = GenerationType.IDENTITY)
    @NonVisual
    Long id

    @ManyToOne (optional = false)
    Blog blog

    @Column (nullable = false)
    @Validate ("required")
    String title;

    @Column (nullable = false)
    @Validate ("required")
    @DataType ("longtext")
    String content

    @Column (nullable = false)
    Date posted
}

The @Validate annotations are Tapestry-specific and are being applied to the fields (allowing this is a new feature in 5.0.12). Because the getters and setters are generated without line numbers, the order of the properties ends up being a jumble; the @ReorderProperties annotation puts them into a reasonable order. You win some, you lose some.

On the services side of things, Groovy is a big win with Tapestry IOC. I have an AppModule to configure a few things, such as turning off production mode (which turns on full exception reports), and I want to override part of my Hibernate configuration in development mode:

package com.nfjs.hls.blog.services

import org.apache.tapestry5.SymbolConstants
import org.apache.tapestry5.hibernate.HibernateConfigurer
import org.apache.tapestry5.ioc.MappedConfiguration
import org.apache.tapestry5.ioc.OrderedConfiguration
import org.apache.tapestry5.ioc.annotations.Symbol

class AppModule
{

    def contributeApplicationDefaults(MappedConfiguration configuration)
    {
        configuration.add SymbolConstants.PRODUCTION_MODE, "false"
    }
  
    def contributeHibernateSessionSource(OrderedConfiguration configuration,

                                         @Symbol ("tapestry.production-mode")
                                         boolean productionMode)
    {
        if (!productionMode)
        {
            configuration.add "DevMode", {conf ->
                conf.setProperty "hibernate.show_sql", "true"
                conf.setProperty "hibernate.format_sql", "true"
                conf.setProperty "hibernate.hbm2ddl.auto", "create"
            } as HibernateConfigurer
        }
    }
}

The ability to wrap a closure as a simple interface (the kind of callback interface used throughout Tapestry) is very, very useful. The only fly in the ointment is GROOVY-2827, which keeps me from using SymbolConstants.PRODUCTION_MODE with the @Symbol annotation on the productionMode parameter.

I think I can get used to the Groovy syntax; I actually prefer the Groovy closure syntax to the Ruby block syntax though I generally prefer the Ruby approach elsewhere.

I expect to write all of this application in Groovy, and that may ultimately become an overall recommendation. Just us crazy framework authors should be coding in pure Java!