Tapestry Training -- From The Source

Let me help you get your team up to speed in Tapestry ... fast. Visit howardlewisship.com for details on training, mentoring and support!

Friday, August 06, 2004

HiveMind happenings

So, by developer vote, SDL (Simple Data Language) has been removed, and the ocean may return to a gentle simmer. I learned a bit about JavaCC with SDL and was excited by it, but I'd rather concentrate on what HiveMind does really well, and not be sidetracked. Instead, we're looking for ways to further reduce the size and complexity of HiveMind's XML module deployment descriptors, and reduce the amount of code in general.

One thing else I've been working on is the ServicePropertyFactory. The idea here is that sometimes service A needs a property from service B to do its work. One approach would be to use service-property:B:foo to inject property foo from service B into service A.

But what if the value can change over time? In Tapestry, examples of this will be access to the HttpServletRequest, HttpServletResponse and HttpSession objects for the current user and current thread. These objects are specific to one thread and change on every request.

Your code could hold a reference to service B and get the property from B every time it is needed. But that's more code, more code paths, more Law of Demeter violations.

What would be nice would be if that property of B could be bound to service A? That's what ServicePropertyFactory does. It creates a service implementation, a proxy. The proxy keeps a reference to service B and, on each method invocation, gets the property from B and re-invokes the method on the property. This is service C, the binding between A and B, and is what gets injected into A.

This allows the code for A to treat a property (say, the HttpServletRequest) as if it was unchanging, when in fact, it is constantly changing. There are pitfalls ... the cost of accessing the methods of the proxy is higher than an ordinary object (it has to go get the real value from the underlying service), and you wouldn't want service A to pull any information out of the property and store it, since that kind of data will likely not be valid after the current request.

What this does do is allow the more dynamic parts of the HiveMind application to be isolated in their own services and allows the rest of the code to stay simple.

Friday, July 30, 2004

Ant best practice: One Compile, One Directory, One JAR

Do yourself a favor. When you are laying out your source tree, follow this rule: One Compile, One Directory, One JAR. Just because Ant supports includes and excludes on its <javac> and <jar> tasks doesn't mean you should actually use them!

A proper source tree structure may have multiple source code roots, and will run javac multiple times -- but each execution will compile all files under a particular source directory into its own, individual classes directory. The contents of that directory (perhaps combined with some deployment descriptors or other resources) will form a JAR. If anything depends on that code, it should add the JAR (not the classes directory, and certainly not the source directory) to its compile time classpath.

Why is the compile-and-split approach dangerous? It makes the build process and build scripts more complicated -- possibly lethally so. It ensures that the build process will not generate class files the same way (and into the same directories) as the IDE. It increases the likelyhood that some classes will be duplicated (compiled multiple times to multiple directories and packaged into multiple JARs), which can easily lead to mysterious ClassCastExceptions at runtime.

A particularly important case involves deployment into an application server. Once you get into complex EAR deployments (with multiple EJB JARs and WARs), the class loader issues can quickly get out of control. By using multiple compiles, you can cleanly ensure that the libraries available at build time match the libraries available to the runtime classloader for that artifact (that EJB JAR or WAR). Without this simple bit of discipline and organization, you can lose big chunks of time to tracking down ClassNotFoundExceptions.

I'm always amazed when I see this kind of thing in otherwise sane code and build environments. I've also seen just how far wrong this approach can go ... such as one source tree, one compile, 40+ JAR files, 10,000+ lines of build scripts!!

.Net vs. Struts ... and the winner is Tapestry!

Just something I noticed on the Cardsharp on Software blog:

Tapestry from the Jakarta Project. Yes, it's not JSF. Yes it's "non-standard." Yes, it requires you to learn a new library and a new way of doing things. But it's better. Far less configuration than Struts, and what is there makes sense. Instead of a jumble of disconnected concepts, the whole library is built around an event-driven mental model that just makes sense. Things that would take me 5 lines of code in Struts take one or two in Tapestry. Imagine being able to tie an event handler in a Java class to a link or a button with no configuration file changes! Think about forwarding to a new page and passing objects to the "action" by setting properties on it istead of hucking everthing into the request or session. Think about how much you like Tiles and imagine what it would be like if everything was a tile, but didn't require all the verbose <tiles:put> lines just to set parameters on your tiles. Think about all those times you'd wished you could put custom configuration information into your <ACTION> tags in the struts-config.xml (especially if you've tried to customize it using the <set-property> nonsense!). Think about how nice your life would be if your ActionForm and your Action were actually in the same class in a way that made sense, and think about how nice it'd be if the framework implemented all the getters and setters for you. Think about implementing complex interactions without jumping through all kinds of hoops to manage the state of your page.

And I'm only scratching the surface here.

Thursday, July 29, 2004

This Great Hacker Does Use Java

Contrary to Paul Graham's misguided treatise on Great Hackers, this great hacker (by his definition, I prefer the term "developer") absolutely does use Java and loves it. You can pry it out of my cold, dead hands kind of love.

Side note: coming from an Objective-C background, I was initially very dismissive myself of Java. I still prefer many aspects of the Objective-C syntax, and Objective-C has killer runtime libraries ... but I've grown to love and be reliant upon Java type safety, and garbage collection. Both are pre-requisites for solving problems in an object oriented way: using large numbers of small, simple objects.

I feel there is a kind of insipid game out there I call "chase the gigahertz". Every time the language (COBOL, then C, then C++, then Java, now Ruby and friends) catches up to the available computing power and the available libraries are in a state where you can actually get something accomplished, there's a lemming-like urge to jump ship and start all over again in the latest and greatest ... with a less efficient, less optimized environment that pushes the performance barrier back up and out of reach.

Wait. Stop. Ask yourself "why?". As the differences between generations of languages become more and more subtle, you have to ask yourself ... what do I gain by switching? What cost do I pay? Am I throwing out the Baby with the Bathwater? Sure, you'll be incrementally more effective at doing the kind of things that look good in demos and tutorials ... but the moment you reach for your handy XML parser and it isn't there, or a friendly GUI toolkit and you have to start from scratch. Database access. Command line parsing. Transaction management. Yes, analogues exist for all of these, but you'll be starting from scratch to learn their APIS, quirks, bugs and workarounds. Just how compelling is that nifty bit of syntax, those couple of characters you don't have to type, that convenient built-in function?

For me, Java hits a terrific sweet spot. Vast numbers of open-source libraries. Very competent language. Minimal amounts of stupid syntax. Of course, there are things I'd change ... but in my analysis, its not about the language syntax its about how you use the language. I'm a framework guy; when I get annoyed at writing dumb, repetitive code I build a framework to take care of that dumb, repetitive stuff for me. I write it once, test it, forget how it works. Tapestry saves me a lot of trouble when dealing with the ugly intricacies of web applications. HiveMind saves me a lot of trouble when building complex apps from lots of simple little pieces. I don't throw out years of effort and hope some new language will solve my problems for me ... I face up to my issues and fix them right here, right now, in Java and be done with it.

Java is good enough. Sometimes "good enough" today is better than "ideal" tomorrow. I can get insanely great things done in Java today, and tomorrow, and the day after.

I'm not saying you should be complacent. Dave Thomas recommends learning a new language at least every year, to give you more experience, more insight into new ways of accomplishing things in your language of choice. I hope the Java language designers to the same, and give us some stuff we could use (such as continuations and closures). As a software craftsman, your technique should transcend any one programming language.

But back to Paul Graham; he's making that tired claim about productivity, as if productivity is linked to characters typed. So typing productivity trumps everything else? As you can see, I don't think so. I think he's fallen prey to a bad analogy ... that because some great hackers chose to use scripting languages such as Perl, Python and Ruby then all productive developers should use those languages for all chores, great and small. He's decided that one language (or class of language) is universally correct, and he's using that as a litmus test for being a great hacker.

Whereas, I think "the appropriate tool for the job", scripting languages for simple, scripting things. Power languages, such as Java, for long term, high quality, maintainable development. Individuals can move mountains, if they have the right tools. For the applications and frameworks I build, the right tools are Java, the Java runtime and the innumerable Java libraries.

German Tapestry Book Update

As mentioned previously, a German language book on Tapestry has been published ... and I now have a copy of it myself (thanks, Stefan!). Sorry, no review ... I don't read German, which means I can't even read my own vorwort (forward).

Saturday, July 24, 2004

Copland: HiveMind for Ruby

From the department of nifty!: Copland (named after the composer) is a Ruby version of HiveMind. Not just similar ... expressly based on HiveMind.

Wednesday, July 21, 2004

Groovestry?

Michael Henderson has been combining Groovy with Tapestry and has some pretty exciting results. He's not satisified with just listener methods in Groovy (using the little known <listener-binding> element) ... he wants page classes in Groovy, and eventually, he wants to eliminate page and component specifications and do those in Groovy as well. I can't think of a reason why not!

Tapestry Goes German

If you read German, you have something new to look forward to: Stefan Edlich and Patrik Kunert have written their own Tapestry book, subtitled Webanwendungen mit dem Apache Framework. A few more details are available at Software & Support Publishers' web site (which includes a downloadable chapter). Stefan emailed me to share his joy at holding a copy in his own hands, something I can relate to (was that only back in March?).

Sunday, July 18, 2004

HiveMind and autowiring

Sometimes the function of a community is to force you in the right direction. With HiveMind, there's the ongoing concern of too-much XML. Erik Hatcher somewhat forced the issue (he can be very incisive) and the response was the controversial Simple Data Language.

However, even that may not have been enough. The SDL is more concise, but your still have a lot to say in it. Over time, the BuilderFactory has been extended, in a number of ways to decrease the amount of SDL you need to put in place. For example, if a service needs to know its service id, it just needs a setServiceId(String) setter method and BuilderFactory will invoke it.

My more recent changes build on a very nice patch supplied by Marcus Brito, which allows you to access a service within the Registry using just its service interface. HiveMind will find the lone service point that implements that service and return it (throwing an exception if no service, or more than one service, implements the interface). Marcus did a good job, supplying a nice patch including JUnit tests (I only needed to do some minor touch ups). I can see how useful the approach is, though there's the danger of it causing problems (what happens at a later date when a second service is introduced?).

I've introduced another form of autowiring into BuilderFactory: after it has set all the properties it knows about (via autowiring and expicit sets) it checks the remaining writable properties. Any such properties that are interfaces are assumed to be services, and the unique service for that interface is located. Again, errors abound when the interface isn't a service, or there is more than one service implementing the interface (an attribute allows autowriting of services to be disabled).

The upshot of this is that you can specify less and less in the module deployment descriptor, and HiveMind will still do the right thing (or, it will provide detailed feedback about what went wrong).

The other big change in HiveMind of late is the concept of object providers. This is (as with much in HiveMind) somewhat abstract but very exciting. Previosly in HiveMind, the schema format identified the type and interpreation of attributes using translators. Is an attribute a configuration id? Then use the configuration translator. Is the attribute a service id? Use the service translator.

This is good as far as it goes, but it tends to promote a kind of propogation of elements. Look at BuilderFactory, which has a bunch of elements for different types: set, set-long, set-service, set-configuration and so forth.

This is also limiting; a property of an object (or service) to be set may be an interface, and it doesn't matter to the object whether that actual instance plugged in is a service or an arbitrary object. Given factory services (such as BeanFactory), its evident that HiveMind services aren't the only interesting objects out there. That's a problem that forces some people to cut-and-paste BuilderFactory to add new ways of supply objects to the constructed service. That's not good!

This is where the new object translator comes it. The translator is driven by a configuration point, ObjectProviders. The configuration defines different prefixes and provides services that know what to do with strings with that prefix.

So, for example, service:MyService will be intepreted as a service id, and bean:MyFactory:name will access a BeanFactory service and ask it for the name bean. There's even a service-property:MyService:propertyName option, to access a service and read a property from it.

For building services, BuilderFactory now has a set-object element. Further, since this is driven by a configuration point, it will be possible to add application-specific prefixes as well! So, once again, HiveMind is object soup, and its getting less important where those objects are coming from.

Saturday, July 10, 2004

Mozilla Gains on Internet Explorer

Slashdot is reporting that Mozilla Gains on Internet Explorer. I can say personally that I have switched entirely from IE to Firefox (0.9.1). Unlike the earlier version I tried (then named Phoenix), Firefox has been very fast, very stable and very ... fun. Just love the tabs.

I keep IE around for occasional compatibility testing, and for the couple of sites I use that don't like Mozilla, such as Shutterfly.

The only thing I miss is InstantSource, but I have contacted the authors about porting it over, and perhaps they will. I'm sure I'm not the only one. The WebDeveloper extension for Mozilla is really good, and has some great features that InstantSource doesn't have, but is not as useful as InstantSource.

Wednesday, July 07, 2004

Updated Tapestry In Action Examples

I just spent an hour or two updating the Tapestry In Action examples. This new source code distribution will be available from Manning's web site, in the meantime you can get it here (3.3 MB).

Changes:

  • Now works with and requires, Tomcat 5 (tested with 5.0.25).
  • examples.war renamed to tiaexamples.war, to not conflict with Tomcat's built-in examples.
  • No longer changes Tomcat's server.xml
  • Now includes the Tapestry 3.0 libraries and dependencies ... no seperate download needed.

Why include the libraries? Because it turns out that the exect version of the Tapestry framework is critical to getting the WAR to deploy. Initially I couldn't get tiaexamples.war to deploy and it took me forever to track it down; there's a reference to the Tapestry JAR in its web.xml:

  <taglib>
  	<taglib-uri>http://jakarta.apache.org/tapestry/tld/tapestry_1_0.tld</taglib-uri>
  	<taglib-location>/WEB-INF/lib/tapestry-3.0.jar</taglib-location>
  </taglib>
If that JAR doesn't exist (say because you tried to deploy using 3.0-rc-1) then the WAR just doesn't deploy. You have to dig through the Tomcat logs for a while before you find the problem. The best solution was to ensure that the version matches up by including all the libraries and dependencies (getting the dependencies being every newbies #1 issue). Because this distribution isn't served off of apache.org or a mirror, including the dependencies is completely legitimate.

Sunday, July 04, 2004

HiveMind and automatic reloading

I gave this another thought yesterday while relaxing on the beach. HiveMind doesn't have a way to redeploy because a descriptor changed. That wasn't on e of the problems I wanted to solver with HiveMind initially.

It's a complicated problem, because the very nature of HiveMind is to intertwine services. Service A may use configuration B that includes a bean obtained from service C that is itself driven by configuration D. If a contribution to service D changes, how does that affect service A? I suppose, in theory, you could do some tricky forward chaining so that a change to configuration D will be identified a propagated forward, invalidating service C and configuration B and ultimately service A. You could then have the proxy for service A somehow obtain a new instance that reflects the change to configuration D.

What are you willing to pay for that? The CPU and memory expense of determining what has changed and what is affected looks prohibitive to me. Something would have to poll for changes to the modules, re-parse the module descriptors, rebuild the various module descriptors, and send notifications to proxies and other objects.

The thing is, HiveMind is supposed to be light and fast. I have always pictured HiveMind running inside an EAR. When the EAR is redployed, the old HiveMind Registry is shutdown and a new HiveMind Registry is created and started to replace it. During development, just bounce your application (i.e., restart Jetty). I'm doing this today on my current project, and it's not a great imposition ... just a couple of seconds to stop and restart. The Jetty Launcher plugin makes it a single click.

So I'm still filing this under YAGNI(You Aint Gonna Need It).

Wednesday, June 23, 2004

Clever Jetty Hack: Dynamically merging directories inside a web application context

I'm working on converting an existing web site from homebrew servlets and JSPs to Tapestry. What's interesting is that the application in question is deployed into different hosts using different "skins". Certain stylesheets and images are different, and there are minor differences in layout and behavior.

So, I'm working with their workspace directory layout; there's a common directory that contains the main content (images, stylesheets, Tapestry artifacts). Then there's additional directories, such as defaultskin, that contain skin-specific assets and artifacts.

When building and deploying, the content is merged together to form a composite web application context.

For development purposes, I want to leave the directory structure intact, but that makes it impossible to run the application (the assets and artifacts in the defaultskin directory simply aren't visible. What I need is to be able to have a virtual directory within my web application that points to an entirely different directory; I want to map /skin in my context to the defaultskin folder, so I can use URLs like http://.../skin/images/about.gif.

Did a bit of research and realized that the correct approach was to create my own implementation of Jetty's WebApplicationContext that included the necessary hooks:

package portal.jetty;

import java.io.IOException;

import org.mortbay.jetty.servlet.ServletHttpContext;
import org.mortbay.jetty.servlet.WebApplicationContext;
import org.mortbay.util.Resource;

/**
 * Used only during testing using Jetty, this allows resources "within"
 * a web application context to be retrieved from an entirely different
 * directory.
 *
 * @author Howard Lewis Ship
 */
public class SkinContext extends WebApplicationContext
{
    private String _skinURI;
    private String _skinPath;
    private Resource _skinResource;

    /**
     * Standard constructor; passed a path or URL for a war (or exploded war
     * directory).
     */
    public SkinContext(String war)
    {
        super(war);
    }

    public String getSkinURI()
    {
        return _skinURI;
    }

    public void setSkinURI(String string)
    {
        _skinURI = string;
    }

    public Resource getResource(String contextPath) throws IOException
    {
        if (contextPath.startsWith(_skinURI))
        {
            String postPrefixPath = contextPath.substring(_skinURI.length());

            return _skinResource.addPath(postPrefixPath);
        }

        return super.getResource(contextPath);
    }

    public String getSkinPath()
    {
        return _skinPath;
    }

    public void setSkinPath(String string)
    {
        _skinPath = string;
    }

    public void start() throws Exception
    {
        super.start();

        _skinResource = Resource.newResource(_skinPath);
    }

}

This gets combined with a specialized jetty.xml (startup configuration file):

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure 1.2//EN" "http://jetty.mortbay.org/configure_1_2.dtd">

<!-- At deployment time, the WAR will be built properly, so that content under 
  "skin" is just appropriate to the type of site. During development, we don't want to have
  to copy files around unecessarily, so we alias the web/defaultskin directory as "/skin". -->
   
<Configure class="org.mortbay.jetty.Server">
  <Call name="addListener">
    <Arg>
      <New class="org.mortbay.http.SocketListener">
        <Set name="port">8080</Set>
      </New>
    </Arg>
  </Call>
  
  <Call name="addContext">
    <Arg/>
    <Arg>
      <New class="portal.jetty.SkinContext">
        
          <Arg>web/common</Arg>
        
        <Set name="contextPath">/</Set>
        <Set name="skinURI">/skin/</Set>
        <Set name="skinPath">web/defaultskin/</Set>
      </New>
    </Arg>
  </Call>
  
</Configure>

Viva open source! I didn't even bother pulling down the Jetty source to figure this out, I just navigated around Jetty's browse CVS to find the few code snippets. Of course, I don't think this solution is ideal for deployment, but that's not the point ... it's about making my development time efficient and I'm quite happy with it.

Tuesday, June 15, 2004

A little EasyMock trick

I had a problem with EasyMock ... I had to validate a call into a method that passed an exception. Exceptions don't compare well to each other so this was causing problems.

Fortunately, EasyMock has an escape clause for this purpose, you can override how it compares expected vs. actual arguments for a particular method. Thus:

public class ExceptionAwareArgumentsMatcher extends AbstractMatcher
{
  protected boolean argumentMatches(Object expected, Object actual)
  {
    if (expected instanceof Throwable)
      return expected.getClass().equals(actual.getClass());

    return super.argumentMatches(expected, actual);
  }
}

This just checks that the exception passed into the mock object is the right type, not that it is any particular instance.

In use, it looks like this:

MockControl c = MockControl.createStrictControl(ErrorHandler.class);
ErrorHandler eh = (ErrorHandler) c.getMock();

eh.error(LOG,
  "Unable to order cartoon character 'wilma' due to dependency cycle:"
  + " A cycle has been detected from the initial goal [wilma]",
  null,
  new CyclicGoalChainException(new Goal("")));

c.setMatcher(new ExceptionAwareArgumentsMatcher());

c.replay();

HiveMind Framework Stack

I've created a new diagram to illustrate the HiveMind framework stack:

I think this is a good, at-a-glance to what HiveMind is doing for you; the Application has pulled a couple of facade services out of the HiveMind Registry, but the implementations of those services are dependent on many other services and configurations inside HiveMind and even on external entities (such as an session EJB inside the J2EE area).

Saturday, June 12, 2004

Order Up at the Tapestry Deli

MindBridge has just put up a new Tapestry web site: http://www.t-deli.com.

Not only does it contain a working version of the Tapestry Workbench, but it contains Tapestry-related Ant tasks and components. A lot of the stuff people are always looking for, like smarter version of Tapestry's Conditional and Foreach components that not only have more sensible names, but also automatically adapt when they are inside a Form.

Occasionally I feel the crushing weight of Tapestry and the needs of the ever-growing community bearing down on me. It is such a relief to see the other committers sharing the load, especially in such a visible way. Thanks MindBridge!

Friday, June 11, 2004

Boston NEJUG Tapestry Presentation

Last night I gave my Tapestry presentation to the Boston NEJUG. Good crowd, about 100 people, at Sun's campus in Burlington, MA. Amazingly, I didn't have my expected projector problems; I was able to face the audience, and type, and see what was getting projected. Makes things so much easier.

It started a bit slow, and I was concerned that I was going to run out of material before I ran out of time. What I didn't realize was that these folks were simply too polite to interrupt with questions; once I asked for questions I got deluged. There were minor hiccups in terms of getting the examples put together (I'm not quite Steve Jobs when it comes to live demos), but those hiccups do demonstrate the support that Spindle and Tapestry give you when developing, so its not a bad thing.

Dan Jacobs (who runs the WebTech Users Group) had a few challenges for me .. such as ripping out the entire Form from the AddAddress page and moving it into its own component. No problem (OK, minor problem, I should have watched the error line reported by Tapestry) and I got it working reasonably in just a couple of minutes.

I've put up an HTML version of my presentation, by request. As my presentation get better, my slide sets get smaller, and that's very good! I'll get a copy of the Eclipse project up shortly.

More on Tapestry and JSF

A recent article at OnJava, Improving JSF by Dumping JSP has been the source of some interesting discussions. The author, Hans Bergsten, is on the JSF expert group and is quite aware of Tapestry ... he even modified the Hangman example from chapter two of Tapestry in Action for this article.

Limitations of JSF/JSP

Interestingly, the JSF component tree is built initially during the JSP render, the first time a page is accessed. That means components early in a page may not be able to reference or interact with components later in the page on that first render, which results in some oddities. He gives an example of a label component and an input field component, where the label does not appear the first time a page is rendered. Whoops! That a violation of the consistency principal to me.

Then it turns out the JSF mandates lots of limitation on how you write your JSP. For example, in most cases, you must use a <h:outputText> tag rather than literal text to output literal text ... otherwise the text tends to end up in the wrong place!

Almost two years ago, I was on a quest to mix and match Tapestry and JSP, such that a Tapestry page could use a JSP instead of an HTML template to render its response. I made a lot of progress but hit some complex cases where I could not merge the two models together, and I instead went outside the box to create Tapestry 3.0 (which has JSP-like features, such as implicit components).

Apparent the JSF team hit the same snags and, because they are blessed by the powers that be, simply said "Thou shalt not do the things that make JSF break. If you do, it is your fault, not ours.". Wish I had it so easy!

Now, as much as the JSF leaders might say that the limitations discussed in Hans' article are merely the result of using JSPs, and that JSF transcends JSPs ... well, any non-JSP implementation of JSF is going to be proprietary, and non-portable or lack the IDE tool support that justifies JSF in the first place. This should sound familiar, its the same set of promises that came out five years ago for EJBs (tools will fix everything!), and the same pattern of complexity, lock-in, and inefficiency that resulted.

Stepping closer to Tapestry

Hans' article continues with a discussion of how to use JSF with a custom ViewHandler that mimics Tapestry's HTML templates and page specifications. That's great I suppose ... but what I'd rather see is a comparison to chapter six which rebuilds the simple Hangman example using custom components. Creating new components in JSF is very involved ... lots of classes, lots of XML, lots of fiddly bits. Components in Tapestry are streamlined and easy ... and powerful, in that they can have their own HTML templates. So as complimentary as this article is to Tapestry, it is also an unfair comparison.

More observations

A JSF application can act as an inversion-of-control container, creating and configuring managed beans. This is a feature of Tapestry as well, primarily through the <bean> element of the page and component specifications. I think this is a better solution, to keep the beans associated with the page (or component) and for the names to be local to the page (or component). A lot of thought in Tapestry has gone into allowing different developers to work on different parts of an application without conflict. Anything global is a measured risk ... in Tapestry, the only true globals are the names of the pages.

It was interesting to the the <component> tags for selectForm and selections, where selections was nested inside selectForm (this also meant that in the template, the selections component was referenced as selectForm:selections. In Tapestry, the two components would be peers (both children of the containing page) and the relationship (that the form encloses the selections) would be determined at runtime, during the render. This is important, because in Tapestry a Form cannot know statically about all the form element components that will be rendered inside its body ... those components may be in the body of the Form (the most typical case), or inside components within the body of the Form ... or, with clever use of Block and RenderBlock, potentially on an entirely different page of the application! Tapestry doesn't care, which is why artifacts such as form control ids must be allocated dynamically.

Now I may be nitpicking here, and I'm not sure which of these is related to the JSF standard, and which to Hans' ViewHandler implementation. The HTML template includes the same kind of previewable HTML as Tapestry, but its not clear how or if it is discarded. Additionally, the javax.faces.Command/javax.faces.Link combination (used to render the link around each letter) has a rendered parameter similar to Tapestry's disabled parameter ... but neccessitates a duplication of the javax.faces.Graphic to cover the cases where the link is not rendered (because that particular letter has already been guessed).

Summary

I just don't see JSF aspiring to any of Tapestry's guiding principals: Simplicity, Consistency, Efficiency, or Feedback. It's very gratifying that the JSF experts are looking to Tapestry for inspiration ... this is quite similar to the situation with the EJB 3.0 specification and Spring.

There has been some discussion of what Tapestry can do that JSF can't. That's a silly question ... there's nothing Tapestry can do that JSF can't. There's nothing either framework can do that a servlet can't, or a Perl script for that matter. It's a question of how easy it is to get it working, and how well it works at runtime, and how maintainable it is in the long run.

Based on everything I've seen, JSF still faces an uphill battle in these areas. Yes, FUD. Yes, I'm biased. But last night at the NEJUG, I coded two simple applications while explaining Tapestry and fielding questions and where I messed up, Spindle or Tapestry itself helped me fix my problems. I doubt I could have accomplished the same things using the JSF RI (even if I had the necessary experience) in the time available. I suspect, over time, we'll be seeing some better shoot-outs between Tapestry, WebWork and JSF. Struts, at this time, is turning into an also-ran kept around for legacy purposes. In the meantime, I repeat the rallying cry: Results Not Standards.

Wednesday, June 09, 2004

New in HiveMind: PipelineService

Just finished checking in the hivemind.lib.PipelineFactory service into HiveMind.

This is neat stuff; pipelines are a close cousin to interceptors, but tend to be things that are a) coded manually and b) specific to a particular service. Think Servlets and Servlet Filters.

The PipelineFactory is a service implementation factory that constructs a pipeline ... a series of filters that call each other, then eventually call an underlying service. It looks like this at runtime:

The bridge classes are fabricated at runtime and connected together with the filters. Each filter is passed the next bridge as a parameter (the last filter gets the terminator instead), and is free to invoke the bridge before or after doing some kind of work, and to change the parameters as it sees fit.

A minimal use of this shows up as a service, a configuration, and some contributions to the configuration:

service-point (id=MyPipeline interface=mypackage.MyService)
{
  invoke-factory (service-id=hivemind.lib.PipelineFactory)
  {
    create-pipeline (filter-interface=mypackage.MyFilter configuration-id=MyPipeline)
  }
}

configuration-point (id=MyPipeline schema-id=hivemind.lib.Pipeline)

contribution (configuration-id=MyPipeline)
{
  filter (service-id=FrobFilter)
  filter (service-id=BazFilter)
  filter (service-id=FooFilter after="*")
}

This will be very useful in Tapestry 3.1 because many different services (not just Tapestry engine services, but all kinds of infrastructure) will be implemented as pipelines of simpler services. These pipelines will be exposed as configurations that will be pluggable. Got a particular transaction management strategy that you want to apply to all requests? Plug into the correct pipeline. Want to add some authorization checks to some kinds of requests? Plug into those service's pipelines.

This concept is demonstrating two key selling points of HiveMind:

  • Libraries can provide configuration points that most applications can ignore, but certain applications can plug into.
  • Mixing services and configurations (services passed as data, or services defined in terms of contributed data) extends the power of post the service model and the configuration model.

Once again, this underscores the difference in philosophies between HiveMind and Spring ... in Spring you still code against the metal. It's a much better metal to code against than the raw Java APIs or the raw J2EE APIs, but to make use of a service provided by Spring, you need to know a lot of details about how and when to configure and instantiate it ... it all goes into your springbeans.xml file.

HiveMind's philosophy is that just by having a framework available on the classpath, it will provide services and configurations that you don't need to know about. You can find out about them via HiveDoc (and other documentation) and take advantage of them as needed.

Take the example above: you don't need to know about the implementation of the PipelineFactory:

service-point (id=PipelineFactory interface=org.apache.hivemind.ServiceImplementationFactory)
{
  invoke-factory (service-id=hivemind.BuilderFactory)
  {
    construct (class=org.apache.hivemind.lib.pipeline.PipelineFactory service-id-property=serviceId)
    {
      set-service (property=classFactory service-id=hivemind.ClassFactory)
      set-service (property=defaultImplementationBuilder service-id=DefaultImplementationBuilder)
    }
  }
}

This configuration, and its dependencies on the ClassFactory and DefaultImplementationBuilder services, are the PipelineFactory's concern. It might change at some point due to some refactoring within the HiveMind framework ... and it won't affect your code at all. You may use some hypothetical third party framework and it may make use of PipelineFactory in its module descriptor and you don't have to know about it.

This is not all to say you couldn't create some kind of pipeline factory in Spring. You absolutely could ... but its a lot of work. It would be easier to create a bunch of individual beans and configure them to each other ... you don't save a lot of effort by adding additional abstractions (though you'd probably have to code a "bridge" class manually) ... you don't save anything because you are responsible for defining and configuring every bean in your application, even when the code comes out of a third party framework.

Creating a pipeline like this in HiveMind is a bit of effort, a new recipe to follow. However, contributing into your own pipeline, or a pipeline provided by another framework, is very streamlined. The way HiveMind can mix and match configurations and services, and the way it can accumulate contributions from multiple modules, is the key to this ease of use.

Saturday, June 05, 2004

Getting ready for pipelines

In between some yardwork around the homestead, I've been coding up a storm on the HiveMind front. This morning, I hit all the remaining code inside the framework to handle the new localized message strategy (more on that in a later post). I then refactored ClassFab and friends to make them easier to use outside of HiveMind. ClassFab is my attempt to "tame" Javassist, which is a powerful bytecode enhacement framework used by Tapestry and HiveMind. ClassFab streamlines the work of creating new classes at runtime ... the heavy lifting is done by Javassist, of course, but ClassFab is a prettier face.

The refactoring allowed me to create a better test suite for ClassFab; previously, the tests were a bit indirect, more like integration tests, where ClassFab was used to create a running HiveMind Registry. I've been trying to get away from that as the primary method of testing in HiveMind (and, some day, Tapestry), based largely on listening to Dave Thomas and reading Pragmatic Unit Testing. I now try to test directly, instantiating the objects inside my unit tests (assisted by EasyMock). I then do an additional integration test or two to verify that my module deployment descriptors are correct. This has a lot of advantages; it's much easier to set up exceptional cases than trying to do so using the integration approach, so this more disciplined approach yields better confidence (and code coverage!)

The main effort was this evening, where I created a pair of related services. The first is passed an interface and it creates a default (or placeholder) implementation of that interface. More Javassist, of course. The second service is a service implementation factory, allowing you to easily create placeholder services.

This is leading up to something very powerful, that builds upon the concept in HiveMind of mixing configurations and services on an equal footing: processing pipelines. Think in terms of servlets and servlet filters. Some series of filters will call each other, and the last filter will call into the servlet itself.

The end result will be a way to easily define a service in terms of a configuration point and a pair of interfaces: the service interface and the related filter interface. The configuration will define a series of filters (with all of the neat ordering power that's already built into HiveMind). HiveMind will dynamically create the necessary classes and such that will make the pipeline operate efficiently. This will be great for Tapestry, as it will break apart a number of monolithic classes and code blocks into tiny pieces, and give end users the ability to easily hook into many different places to provide application specific extensions.

Thursday, June 03, 2004

HiveMind vs. Spring: Philosophy

I was struck this morning with just the right way to explain the difference between Spring and HiveMind.

Spring is based on a recipe philosophy. If you want a particular kind of behavior, there's a recipe for getting it. You want declarative transactions? You lookup up the recipe for it ... you define your bean implementation (in Spring terminology, a "target bean", the other beans you need (a transaction manager bean in this case), and then combine them into a final bean, ready for use in the application.

Along the way, you may have to cut-and-paste some details, particularly the beans (transactionManager and friends) provided by Spring. On the other hand, you have a single place (your beans XML file) to look for all the details.

HiveMind uses a very different philosophy: a component oriented approach. In HiveMind, modules (that is, JARs packaged with a HiveMind module deployment descriptor) provide services (and configurations) using well-known names (those fully qualified service and configuration ids). Just by having the JARs on the classpath, the services are defined and waiting to be used, directly or indirectly, in your application.

Is this difference that important? I think so. HiveMind has a greater opportunity to be an integration platform than Spring. Over time, underlying frameworks change and its easy to imagine a number of situations where a commonly used bean, such as a transaction manager in Spring, would change over time. These changes could be something as simple as adding new properties or methods, or something more dramatic such as renaming a class or moving it to a new package.

Under the Spring model, upgrading from one release to another may break existing bean configurations.

Under the HiveMind model, the provided service would still have the same fully qualified id and service interface even if its implementation is radically changed ... and any such changes are restricted to the JAR and its module deployment descriptor.

In practice, this is not such a big deal; people don't go willy-nilly changing versions of important frameworks for the applications without the expectation that they'll need to do a good amount of testing and perhaps some tinkering.

In fact, the core difference in the philosophies is not about managing change over time, but in managing configurations across different simultaneous deployments. For Vista, the product that birthed HiveMind, there were pending requirements to support multiple databases (Oracle, MS SQL Server, perhaps PostgreSQL) with a single code base. The idea was that the application code would call into a data access object layer (implemented as HiveMind services), and those services would, in turn, call into a database abstraction layer to do the real work. The abstract layer would also be HiveMind services, with a set specific to Oracle, to SQL Server, etc.

One option was to isolate the different abstraction layers into different JARs and just control which JARs were put into deployment.

A pending (and only potential) requirement for Vista was to be able to provide different scaled versions of the product to different clients. So, a small school might get the JBoss version with a small number of Vista tools. The Vista application is divided up into a large number, about 30, different tools, each focused on a particular part of the application, such as Mail, Discussions, Quizzes, Lab Assignments, and so forth.

A large implementation of Vista might want the fully suite of Tools, and be set up to deploy into WebLogic and Oracle.

The HiveMind vision is that the installer tool for Vista could simply decide which of the modules should be deployed based on licensing information. In this way, rather than maintain different installers and different configurations of the product (with potential duplication of code across source trees), we rely on HiveMind to integrate the application on the fly, at startup.

Alas, I switched over to consulting long before I had a chance to see any part of this vision take root at WebCT ... but if you are curious why HiveMind is the way it is, that vision is the root cause.

Wednesday, June 02, 2004

HiveMind is one year old!

Just checking the CVS access logs; I did the first check in of HiveMind code into the Apache CVS on May 30 2003. HiveMind is over one year old now.

Tuesday, June 01, 2004

Spring Integration in HiveMind

I've added some basic integration with Spring to HiveMind. You can now make Spring responsible for createing a service and, effectively, link to it from within HiveMind. That is, the SpringLookupFactory can create a core service implementation by accessing a bean defined within a Spring BeanFactory. This opens up access to all sorts of Spring functionality (especially Hibernate integration) but keeps the syntax simple (and in SDL). Rod has suggested that HiveMind somehow "use" Spring's transaction interceptor, but that's a project for another day.

We'll have to see if the Spring guys follow suit and provide some integration in the other direction; I would love to see something symmetric, where a Spring factory could defer bean creation to a HiveMind service. That would allow Spring beans to tap into HiveMind's much more advanced configuration management.

I've also been wasting time fighting with Forrest to get the menus for the HiveMind site generated properly. It's doing the right thing for non-index.html pages. All the index.html pages keep getting all the menus, not just the menus for the selected tab. Probably a bug deep, deep, deep inside Forrest.

Friday, May 28, 2004

HiveMind home page updated

I've put up the updated HiveMind home page, which now has the Forrest look to it, replacing the old Maven look.

Forrest has its own issues, but at least those issues are localized to documentation, mostly navigation. For example, I just could not get the tabbing thing working; I'd like each of the modules (hivemind and hivemind.lib) to be its own tab, but after struggling for a long time, I've decided to punt. Forrest falls very far short on simplicity, consistency, efficiency and especially feedback ... but at least I can get pretty much the results I want, the way I want them, which I could not manage using Maven. The Maven guys blame Jelly but it's all the same to me!

Despite some pointers to other sets of Ant build tools, I've continued to develop my own home brew stuff, and it's fitting my needs quite well. Even when the rest of HiveMind goes into beta, the build scripts will still be alpha for a while.

Thursday, May 27, 2004

Tapestry Test Assist

A frequent criticism of Tapestry, from the point of view of the Test Driven Development crowd, is that Tapestry is too hard to test ... because all your classes are abstract.

As a stop-gap measure, I've finally gotten around to creating Tapestry Test Assist. This is a simple class, that can be used inside test suites, to instantiate abstract Tapestry pages and components (or any other abstract class, for that matter).

Like Tapestry itself, the AbstractInstantiator will create new fields and methods in the subclass. Unlike Tapestry, it isn't driven by an external specification; it just finds each property that is abstract (i.e., has an abstract getter and/or setter method) and implements the property in a subclass, with a field and pair of accessor methods. Unlike Tapestry, these accessors are very simple, with no hooks into Tapestry persistent page property logic ... and that's fine for testing.

The source code is available as a zipped-up Eclipse workspace. The easiest thing is just to copy the couple of source files into your own test suite. Eventually, this will be part of the actual Tapestry distribution.

Tapestry in Action on java.net

Coincidentally, java.net is also running a discussion of Tapestry in Action. I don't think they are giving away a clopy of the book. I'll be monitoring this forum as well as the JavaRanch. The java.net discussion runs for a month.

Examples from NFJS Denver

I finally remembered to upload the examples from my presentations at NFJS. This is a ZIP file of the Eclipse project. Expect this to change over time as I add more examples for different situations (I also use this application with clients).

Tuesday, May 25, 2004

JavaRanch Radio - Giveaway of "Tapestry In Action"

Just a note:JavaRanch Radio - Giveaway of "Tapestry In Action"

I'm monitoring the forum and answering questions. Any additional help and postings by the Tapestry "faithful" would be most welcome.

Monday, May 24, 2004

Tapestry at NFJS Denver

Just got back from the Denver No Fluff Just Stuff, where I gave two Tapestry presentations and a HiveMind presentation. Matt Raible attended the basic Tapestry session and was impressed with Tapestry.

I didn't stick to my presentation at all; Before the session, I took my finished examples application, gutted the two pages (Login and AddAddress) back down to plain HTML, and either trimmed or removed the page specifications and the Java classes. I then put them back together live! during the session.

Most people really liked this, they saw how quickly the exception report page gets you to fix problems, and just how little goes into the HTML (and the page specification and the Java class for that matter). One evaluation claimed that "watching someone code is boring", but that's the exception to the rule ... the clueful people could see how easily Tapestry would fit into their development cycle, which is the whole point.

Also got to demo some great features in Spindle while I was at it. I did have some stumbling points ... mostly the same problem getting me Dell laptop to work with Jay's projectors (I can't synchronize my screen to the projected view, so I have to code while staring up at the screen). Matt also suggests creating templates for the code, rather than laboriously typing in everything and that's a great idea ... I'll just create templates for each method I'll add to each of the classes.

Had a little fun on the expert panel ... David Geary was pretty cantankerous about JSF vs. Tapestry, and brought out that tired line about "It's a Standard". Results Not Standards folks! More comments on this subject later ...

My other two sessions were underattended ... right now, competing against sessions on Spring and Groovy is a non-starter. In fact, for the Tapestry components session, we just gathered around my laptop, which was a lot of fun.

I'm going to be retooling my presentations and hopefully will have a Tapestry and Hibernate session ready soon. In addition, we will probably combine the two Tapestry sessions together into a "Tapestry Kickstart" double (three hour) session.

Friday, May 21, 2004

JAM -- Another alternative to Maven

I've been learning a lot about Ant 1.6 features, but others may have beaten my to the punch: JAM seems to do all the things I'm planning and, of course, it already exists. Need to check it out a bit more carefully, but if it's less work ... it's less work!

Moving Away from Maven

I've gotten some comments asking why I'm moving away from Maven.

I started to use Maven initially as part of the HiveMind experiment. Fundamentally, I liked two specific features:

Maven does those two things pretty well, though the documentation part has a large number of bugs that makes keeping the documentation up-to-date problematic.

Anyway, measured by my Four Principals, Maven falls shorts:

  • Simplicity - this isn't even on the horizon in Maven-land. If you deviate even a tiny bit from Maven's one-true-path, you are lost! The complexity of plugins, class-loaders, XML documents that are Jelly programs in disguise, lack of documentation, etc., etc. means that doing something trivial can take hours of guesswork. Some parts of this "release candidate" have obviously not even been given a cursory test. Certainly the internals of most plugins are a maze of Ant, Jelly, properties and such, seemingly without end.

    This could be somewhat addressed by documentation, but there is a pitiful amount and what there is, is out of date. Understanding multi-project (the whole point of Maven you would think) is a total challenge, addressed only by endless experimentation.

  • Consistency - I guess this is hard to guage; do you write your own plugin? Write ad-hoc Jelly script? Write and use an Ant task?
  • Efficiency - Maven is sluggish, chews memory, tends to repeat operations needlessly. I could comment on the volume of downloads that occur (for plugins, for libraries the plugins depend on) but that actually isn't an issue, after you run Maven the first time. However, for HiveMind, I've taken to leaving the room while I perform my dist build ... and it's only two small projects.

    One concrete example: when using Maven, I could only get my unit tests to work by turning fork on. With Ant, I'm able to run the unit tests without forking. That's a huge difference.

  • Feedback - Can you say "NullPointerException"?

What I've accomplished in two days using Ant 1.6 will serve me well for HiveMind and for Tapestry ... and beyond. I think we'll be able to get a significant amount of Maven's functionality in a small, finite, understandable package, and be able to use it on the vast majority of pure Java projects.

For example, Here's the build.xml for the framework:

<project name="HiveMind Framework" default="jar">

	<property name="jar.name" value="hivemind"/>
	<property name="javadoc.package" value="org.apache.hivemind.*"/>

	<property name="root.dir" value=".."/>
	<import file="${root.dir}/common/jar-module.xml"/>
	<import file="${common.dir}/javacc.xml"/>								
					
	<target name="compile">
		<ibiblio-dependency jar="commons-logging-1.0.3.jar" group-id="commons-logging"/>
		<ibiblio-dependency jar="javassist-2.6.jar" group-id="jboss"/>
		<ibiblio-dependency jar="werkz-1.0-beta-10.jar" group-id="werkz"/>
		<ibiblio-dependency jar="servletapi-2.3.jar" group-id="servletapi"/>				
		<ibiblio-dependency jar="oro-2.0.6.jar" group-id="oro"/>
		<ibiblio-dependency jar="log4j-1.2.7.jar" group-id="log4j"/>
				
		<ibiblio-dependency jar="easymock-1.1.jar" group-id="easymock" use="test"/>
			
		<run-javacc input="${javacc.src.dir}/SimpleDataLanguage.jj" package-path="org/apache/hivemind/sdl/parser"/>
		
		<default-compile/>
	</target>

</project>

And here's the the project.xml, project.properties, and maven.xml.

I hate to bash the Maven project ... but what I see in Maven is a good, simple, core idea that's spiralled down the wrong path by trying to be everything to everyone. That's a lesson I'm taking to heart as I build something that suits my needs better.

Thursday, May 20, 2004

Maven-like downloads for Ant

One of the key features of Maven that I like is that it will download dependencies for you automatically.

I'm just starting to convert HiveMind from Maven back to Ant and this was very hard to do. I want to download a file, just if it doesn't exist locally (or is out of date), compute the MD5 sum while it downloads, and compare that to an MD5 sum stored on the server.

There was an existing project, greedo that may have done some or all of that ... but it has stalled in that "nearly-done" open source state so many projects reach. No activity in the last nine months. Broken home page. No documentation. I got it to build, but I had to hack their broken Ant build files. Also, I think the <macrodef> features of Ant 1.6 trumps a lot of the functionality in greedo, and I wanted more flexibility with respect to where I get files and how they are stored.

Anyway, I took a peek at the existing Get task of Ant and created a Grabber task. That's a start and it will be necessary in order to build HiveMind in the future. For the moment, it is available as http://howardlewisship.com/downloads/AntGrab.zip. This includes source and a JAR, ant-grabber.jar, that must be placed in ANT_HOME/lib.

A request has come in to discuss how it is used. Now, Grabber is super-alpha, but here's a portion of my Ant-based build environment to demonstrate how it is used:

	<available classname="org.apache.ant.grabber.Grabber" property="grabber-task-available"/>
	<fail unless="grabber-task-available" message="Grab task (from ant-grabber.jar) not on Ant classpath."/>
	
	<taskdef classname="org.apache.ant.grabber.Grabber" name="grabber"/>

	<!-- macro for downloading a JAR from maven's repository on ibiblio. -->
	
	<macrodef name="download-from-ibiblio">
		<attribute name="jar" description="The name of the JAR to download."/>
		<attribute name="group-id" description="The Maven group-id containing the JAR."/>
		
		<sequential>
			<mkdir dir="${external.lib.dir}"/>

			<grabber
				dest="${external.lib.dir}/@{jar}"
				src="${maven.ibiblio.url}/@{group-id}/jars/@{jar}" 
				md5="${maven.ibiblio.url}/@{group-id}/jars/@{jar}.md5"
				/>


		</sequential>
	</macrodef>

Later I use the macro as follows:

		<download-from-ibiblio jar="commons-logging-1.0.3.jar" group-id="commons-logging"/>
		<download-from-ibiblio jar="javassist-2.6.jar" group-id="jboss"/>
		<download-from-ibiblio jar="xml-apis-1.0.b2.jar" group-id="xml-apis"/>
		<download-from-ibiblio jar="servletapi-2.3.jar" group-id="servletapi"/>
		<download-from-ibiblio jar="werkz-1.0-beta-10.jar" group-id="werkz"/>
		<download-from-ibiblio jar="oro-2.0.6.jar" group-id="oro"/>
		<download-from-ibiblio jar="easymock-1.1.jar" group-id="easymock"/>
		<download-from-ibiblio jar="log4j-1.2.7.jar" group-id="log4j"/>

Wednesday, May 19, 2004

Comments enabled for the blog

I've enabled comment on my blog ... not exactly sure how Blogger implements comments, so we'll see what happens! I think you need to be registered with Blogger to post (I'd really not like to see my blog filled up with penis-enlargement ads).

Why separate bin and src distributions?

Something that struck me as I was preparing the latest HiveMind release just now. Why do we in the open-source world bother with separating the binary and source distributions?

Take HiveMind. The binary distribution follows standard procedure: it includes all sorts of documentation. Because of the use of Maven, the documentation set is out of control, but even so, what we have is a 281KB (uncompressed) JAR distributed inside 16,526KB (uncompressed) of documentation. Meanwhile, the source code is just another 1,257KB (uncompressed).

The binary distributions are 3.1MB/1.5MB (.zip vs. .tar.gz) and the source distributions are 556KB/229KB. In other words, adding the source to the binary distribution would not be particularly noticeable ... just an additional second or two at broadband speeds.

If I had my say (which, count to think of it, I largely do) I would produce a combined binary/src distribution and have the documentation as the add-on. A combined binary/source distribution would be approximately 50%/100% larger (since the JAR file is already itself compressed). If you assume that most people download the binaries and source together but largely read the documentation on-line (at least until they get serious about a package) ... then a combined bin/src distro is a win.

Certainly when I've used other packages, I've wasted a lot of time unpacking the binary distribution, using the jar, then having to get the source jar and connect it up inside Eclipse to I could actually debug code that uses the library.

This approach would be better for slow connection users as well; they would get what they need to work (the binary and the source) and could cherry pick the documentation they need from a live web site. Certainly, anyone serious about a package would want the full documentation on their own hard drive ... but why pay that cost just to take a peek? Distributing binaries with (full) documentation makes every user pay that download cost ... or keeps some users from bothering to evaluate the package at all.

It's open-source. The point is to buck tradition and think for ourselves.

HiveMind 1.0-alpha-5

I've just tagged the release, and will have downloads available shortly.

Lots of cool stuff between alpha-4 and alpha-5.

  • Simple Data Language
  • Improved HiveDoc
  • Initializable interface is gone, replaced with an initialize-method attribute on the construct element passed to BuilderFactory
  • Some minor renames and refactorings ... more work to seperate the "public face" (in org.apache.hivemind) from internals that user code shouldn't care about
  • Ability to define service models via hivemind.ServiceModels configuration point
  • Ability to define translators via hivemind.Translators configuration point
  • hivemind.Startup extension point for executing code when Registry is constructed
  • hivemind.EagerLoad extension point for forcing services to be instantiated early
  • Registry.cleanupThread() as a convienience for invoking ThreadEventNotifier

I believe HiveMind is ready to go forward; I would like to see a short beta period and a ramp up to GA release. During that period I hope to devote some time to converting from Maven to Ant and Forrest. Fixing up various link errors in the Javadoc would be nice as well. Generally, documentation is in excellent shape.

HiveMind will currently do everything I need it to do for Tapestry 3.1. That's *my* standard.

There are still a few debates out there; I've seen people strongly pro-and-con SDL, XML, Scripting ... but nobody's stepped up to the bat to do any work or even provide a really solid proposal. I'm still strongly in the declarative vs. procedural camp (i.e., no scripting) and vastly prefer SDL syntax to XML syntax.

Learning to love EasyMock

I've finally started using EasyMock with the HiveMind testing ... that's a huge amount of power in a tiny, little package!

If you haven't heard of this, the idea is that you can create mock implementations of services easily. First you create your control, and obtain the mock from it.

Next, you "train" your mock object, by invoking methods on it. The mock object and the control work together to remember the order of methods you invoke, and the argument values passed in. You use the control to specify return values.

Finally, you use the control to replay the mock and then test like it a real object.

Here's an example for the HiveMind suite:

    public void testSetModuleRule()
    {
        MockControl control = MockControl.createStrictControl(SchemaProcessor.class);
        SchemaProcessor p = (SchemaProcessor) control.getMock();

        Module m = new ModuleImpl();
        Target t = new Target();

        p.peek();
        control.setReturnValue(t);

        p.getContributingModule();
        control.setReturnValue(m);

        control.replay();

        SetModuleRule rule = new SetModuleRule();

        rule.setPropertyName("module");

        rule.begin(p, null);

        assertSame(m, t.getModule());

        control.verify();
    }

Here I'm testing a SetModuleRule, which is dependent on the SchemaProcessor. SchemaProcessor are complex to create, and tied into the whole framework ... a lot of work for the two methods that the SetModuleRule will invoke on it!

This is great stuff, because it lets me easily mock up parts of the framework that are normally pretty inaccessible. Some of my tests use two or three mock/control pairs. This is still a big improvement over my existing approach, which is to feed a HiveMind module descriptor into the framework and test that it does the right thing. That's important, but its more of an integration test than a unit test ... it can be hard to tell precisely what failed.

Monday, May 17, 2004

HiveMind -- ready for beta?

I've done some moderately involved refactorings of HiveMind lately and, in my opinion, everything is just about in place for HiveMind to go beta. I want to clean up some stuff in the Registry, RegistryInternal, Module triad of interfaces, and that will allow me to add a configuration point for eagerly (instead of lazily) initializing services. But onces that's in place, I think its finally time to move from rapidly adding features to finding gaps and fixing any holes. I don't think there are going to be too many (famous last words), but really, I've been writing tests and keeping documentation 95% up to date right on through the process. I want a stable HiveMind so that I can get more work done on Tapestry 3.1.

HiveMind work

Squeezed around the edges of my work in Germany, I got a bunch of work on HiveMind done. I've been doing a bit of refactoring, moving code around and splitting the Registry interface into two interfaces (Registry and RegistryInternal).

I changed <configuration-point> and <service-point> to not take a <schema> (or <parameters-schema>) element, but instead have a schema-id and parameters-schema-id attribute. I then made <schema&;gt; top-level only, and made its id attribute required. I found that a bit ponderous, though, and made changes to allow <schema> inside <configuration-point> (without an id attribute, and likewise for <service-point>/<parameters-schema>). So you can do it "in place" or "top level" but not mix the two.

Some big improvements to HiveDoc as well. The new HiveDoc splits the documentation across more files; separate files for each top-level schema, each service-point and each configuration-point, as well as for each module. Much less cluttered.

I also did an experiement; I copied-and-pasted the hivemind.sdl descriptor 26 times (as a.hivemind.sdl, b.hivemind.sdl, etc.) to see how well the XSLT would go with a fairly large input. The combined registry.xml (built by reading and combining all the descriptors) was half a megabyte but the generation of HTML was still under five seconds (to generate about 2.5 MB of HTML).

Since I was on the road, only some of this has been checked in. I'm in the middle of adding some more AOP-lite functionality; the ability to choose, with the LoggingInterceptor, which methods get logging. It'll look something like:

interceptor (service-id=hivemind.LoggingInterceptor)
{
  include (method="get*")
  exclude (method="*(foo.bar.Baz,int)")
  exclude (method="set*(2)")
  include (method="set*")
  exclude (method="*")
}

This will cause all methods with names starting with "get" to be logged, as well as most methods starting with "set". Methods with certain parameters, or a certain number of parameters, will be excluded.

Back from Germany

Just back from a quick visit to Germany and startext, an IT shop that is getting heavily into Tapestry. They brought me out for 3 1/2 days of training and mentoring and it was a blast. We went through my available presentations quickly, but the fun started with live coding ... by them and by me. They learned a lot about Tapestry and I learned a lot about teaching Tapestry ... such as, dive right into the code as fast as possible!

We hit a lot of subjects quickly, getting right into things like creating new components, and generating JavaScript dynamically. They had some interesting requirements, such as having disabled text fields submit anyway (we had to hook the form's onsubmit event handler to re-enable the fields just before the form submitted). Then things got even wackier when we tried to combine that, with ValidFields using client-side validation, and a drop-down list that forced a page refresh (and caused a second drop-down list to update to a different set of values). In fact, some of the stuff I learned can be rolled into Tapestry 3.1.

I felt bad that I didn't have a lot of time to study up on Tree and Table; as it turns out, they really liked seeing me puzzle it out as I went; they picked up from me some tips about how to do it themselves. All in all, a succesful trip, but getting to Bonn and back (via train and jet) was brutal (and took longer than I had power in my iPod).

Now I'm home for a couple of days to try and resynchronize my internal clock, then off to Denver.

By Request: How Line Precise Error Reporting is implemented

Hi Howard,

I read your blog and I'd like to make a request.  I'd really like to read 
more technical details on techniques you might use to get line precise 
error reporting.

This is one of the best features of tapestry and if you can pass on some of 
the details of how to go about it and get people excited about doing it 
themselves that could only be a good thing.  This is typically an area 
where most other open source projects fail badly.  The normal error 
reporting seems to be the null pointer exception.

Regards,


Glen Stampoultzis
gstamp@iinet.net.au
http://www.jroller.com/page/gstamp

I agree with all of this, but it's not just open source projects which fall flat in this area ... and this is a vitally important area: Feedback, one of my four key aspects of a useful framework (along with Simplicity, Efficiency and Consistency). Without good feedback, the developer will be faced with a challenging, time-consuming puzzle every time something goes wrong in the framework code. It's not enough to say "garbage in, garbage out" ... if the framework makes getting the job done harder, or even just makes it seem harder, then it won't get used, regardless of what other benefits it provides.

Line precise error reporting is not magic, but it is a cross-cutting concern (Dion is thinking about how to make it an aspect), so it touches a lot of code.

It starts with the Resource interface, which is an abstraction around files; files stored on the file system, at a URL, or within the classpath. Tapestry extends Resource further, adding the concept of a file within a web application context. The Location interface builds on this, combining a resource with lineNumber and columnNumber properties.

The XML, HTML and SDL parsers used by both HiveMind and Tapestry carefully track the location (in most cases, by making use of the SAX Locator to figure out where in a file the parser currently is).

All the various descriptor (in HiveMind) and specification (in Tapestry) objects implement the Locatable interface (having a readable location property), or even the LocationHolder interface (having a writable location property), typically by extending from BaseLocatable. As the parsers create these objects, they are tagged with the current location provided by the parser.

Later, runtime objects (services and such in HiveMind, components and such in Tapestry) are created from the descriptor/specification

You can see how my naming has been evolving. Descriptor is a better term, I don't remember where "specification" came from, but it's now entrenched in Tapestry terminology.
objects. The runtime objects also implement LocationHolder, and the location of the descriptor object is copied into the location property of the runtime object.

The next piece of the puzzle is that exceptions need to have a location as well! When an exception occurs in a runtime object, the runtime object throws an ApplicationRuntimeException that includes the correct location.

There are a couple of utility methods on the HiveMind class used to help determine what the correct location is:

    /**
     * Selects the first {@link Location} in an array of objects.
     * Skips over nulls.  The objects may be instances of
     * Location or {@link Locatable}.  May return null
     * if no Location can be found. 
     */

    public static Location findLocation(Object[] locations)
    {
        for (int i = 0; i < locations.length; i++)
        {
            Object location = locations[i];

            Location result = getLocation(location);

            if (result != null)
                return result;

        }

        return null;
    }

    /**
     * Extracts a location from an object, checking to see if it
     * implement {@link Location} or {@link Locatable}.
     * 
     * @returns the Location, or null if it can't be found
     */
    public static Location getLocation(Object object)
    {
        if (object == null)
            return null;

        if (object instanceof Location)
            return (Location) object;

        if (object instanceof Locatable)
        {
            Locatable locatable = (Locatable) object;

            return locatable.getLocation();
        }

        return null;
    }

This method is particularly handy to the ApplicationRuntimeException class, since it may want to draw the location from an explicit constructor parameter, from a nested exception, or from an arbitrary "component" associated with the exception:

    public ApplicationRuntimeException(
        String message,
        Object component,
        Location location,
        Throwable rootCause)
    {
        super(message);

        _rootCause = rootCause;
        _component = component;

        _location = HiveMind.findLocation(new Object[] { location, rootCause, component });
    }

That's pretty much all there is to it ... but it's all for naught if all this location information is not presented to the user. The location will generally "bubble up" to the top level exception, but you still want to be able to see that information. Tapestry's exception report page does a great job of this, as it displays the properties of each exception (including the location property), and then tunnels down the stack of nested exceptions (this is actually encapsulated inside the ExceptionAnalyzer class).

Line precise error reporting isn't the end all of Feedback. Malcolm Edgar, a one-time Tapestry committer, has been working on his own web framework (I believe for internal use at his company) ... it goes one step further, actually displaying the content of his equivalent to an HTML template and highlighting the line that's in error. That's raising the bar, but perhaps Tapestry will catch up to that some day.

Further, simply reporting locations isn't enough. If I pass a null value into a method that doesn't allow null, I want to see a detailed exception (You must supply a non-null value for parameter 'action'.) rather than a NullPointerException. A detailed exception gives me, the developer, a head start on actually fixing the problem. Explicit checking along these lines means that the location that's actually reported will be more accurate as well, especially considering that there's no way to attach a location to a NullPointerException.

Monday, May 10, 2004

Thinking about Flex

I was very impressed by the Flex presentation at TheServerSide.

Flex is a rich client toolkit. Flash and ActionScript on the client. Java and XML on the server. RMI or Web Services in between. The client contains all the state, the server is wonderfully stateless.

I first learned a bit about Flex from Matt Horn, who works for Macromedia; they hired a bunch of J2EE developers in one of their acquisitions, and none of them took to Flash. Flex is their take on how to have their cake and eat it too. It's another XML-based user interface scripting language but seems to have a lot more going on inside, especially with respect to data binding and client-server communication.

Here's a good intro to it: Building Rich Internet Applications with Macromedia Flex: A Flash Perspective.

I've been browsing the docs and playing with the samples. It was easy enough to set up inside Eclipse; I just created a new project, created a content folder, and expanded the flex.war into the content folder; this gives a web.xml, and the necessary libraries and such. Next, I used Geoff's Jetty Launcher to serve up the context folder and could start creating the .mxml files. Flex uses servlet filters to converts .mxml into .swf (Flash movies) in much the same way that .jsp files are converted into compiled Java classes.

Running this on my laptop (Dell Inspiron 8200, 512mb ram, Pentium 4 2gz) generally worked well; yet many key presses had a noticeable (though minute) delay ... but at the same time, there was also a good amount of fading, zooming, sliding images. It's as if user-input in Flash is given lower priority. Anyway, it was all still quite impressive and I expect to do more experimentation.

The documentation I've read so far was very good. Rich hypertext and PDF and lots of detail and examples. If I can get my head around the data binding and communication to the server, I could probably put stuff together right now. The default skin is clean and simple. The default components do a better, easier job of creating simple, clean interfaces than Swing or AWT (though the XUL variants there probably help). Again, it isn't just Flash and ActionScript and XML ... the data binding was given significant thought (I'll be able to tell if that thought was worthwhile at some point soon).

My initial reservations:

  • All your client-side logic is written in ActionScript (really, ECMAScript). Unit testing this is going to be at least as much of a challenge as unit testing Tapestry pages.
  • There's some form of debugger for ActionScript, but it doesn't look integrated ... it looks like it might be command-line oriented.
  • Applications are ultimately monolithic; you can break your .mxml files into smaller pieces, and create components as well (a clever use of XML namespaces), but there's just the one application object. How long is the initial creation of the .swf file for complex apps? How much of that .swf must be downloaded before the user sees the initial page? How will teams of developers work together?

So why is the Tapestry guy interested in this stuff? Because HTML is, ultimately, a dead end. Tapestry squeezes the most out of HTML that can be done, but I firmly believe that in three to five years, some form of rich client will supplant HTML for the zero-delivery cost web applications that are currently created using Tapestry, JSP, ASP or whatnot. I predict that a lot of stuff now down exclusively with HTML will be done using a mix of HTML and Flex (or whatever client-side technology emerges, should Flex fail). This could be good news for Tapestry ... the sites I envision dominating the market will consist of "boutique" HTML (HTML created by non-Java developers) and Tapestry shines at integrating that kind of HTML into a dynamic application. The HTML parts of applications will be much more focused on readable documentation (news sites, some form of community sites, blogs and the like) where the ability to print and book mark are important. The kind of applications currently shoe-horned into the HTML world (help desks, all kinds of corporate infrastructure, CRUD applications) will be easier and better using a rich-client alternative.

At TSS, people were dismissive because of licensing costs ($25K per server, give or take). Well, licensing costs come down (think WebObjects, which went from $50,000 to $750). And software licensing is the smallest piece of the development cost compared to developer time and hardware.

More of concern is the single-vendor aspect. This is anathema in the Java world ... and there's the looming possibility of Microsoft buying Macromedia and killing Java support within it. I don't know what the solution to this is ... perhaps Macromedia needs to open-source Flex with a mixed GPL/proprietary license like Sleepycat's ... for-profit users pay Macromedia, non-profit don't. Alternately, seed flexible APIs into the product to ensure that third-parties will be able to provide Java support regardless of ownership of Macromedia and Flex.

In any case, the concept is exciting, regardless of which vendor finally makes it all work. Factoring out the presentation layer from web-deployed applications would be a great thing in differentiating J2EE from .Net. A stateless server-side (mated to a richly stateful client-side) means that simple, efficient solutions (based on HiveMind, Spring, and Hibernate) will have their power multiplied.

A short break between TSS and Germany

So I'm back from TheServerSide Symposium and have just enough time to catch my breath before heading out to a multi-day engagement in Germany.

I was "in character" even before I arrived in Las Vegas, kibitzing with a group of developers across the aisle from me on the flight in.

My sessions, as well as a "TSS tech talk" video shot went pretty well. The HiveMind presentation still went a bit roughly; I'm beginning to think that, up against the Goliath of Spring, my little David had better differentiate itself quickly ... the distributed configuration (for both data and services) is the key distinguishing feature, and a solution that mixes HiveMind with Spring is a likely winner.

Much more interest in the Tapestry presentation, which is the advanced one convering component creation. Again, this is really an area where Tapestry differentiates itself most strongly from the other similar frameworks (if such things exist). I did a bit more live presentation, which was a chance to show and discuss Spindle and the great Tapestry exception reporting page (and line precise error reporting).

Both sessions filled the small room I was in; I counted about 55 attendees in each session. The Tapestry session might have overflowed that room had it now been up against the very contentious "What's in EJB 3.0?" talk.

I thought the sessions I attended were very good. The keynotes were, alas, ignorable (except for the Flex presentation, which rocked). Most presenters were quite good; I think Rod Johnson did a very good job and was quite gracious towards me personally and towards HiveMind, to the point of quoting my HiveMind presentation during his "J2EE without EJB" session. We also talked frequently between sessions; overall, we respect that our frameworks address different needs and that combining them should be made as painless as possible. Meanwhile, I'm jelous that he's in a position to use his framework in production, something that identifies problems and limitations fast. It really underscores how I was languishing at WebCT; I need to get involved in some real work on a real project and be in the technical architecture driver's seat again; it's been too long.

I talked to so many people over the course of a few days and lost track of it all quickly ... I have to start jotting down notes on business cards. I know I promised copies of the book to Jason Carreira and Kito Mann, anybody else had better send me a reminder! Also, a lot of people are really pushing for Tapestry to support the Portal API somehow, someway.

People have been talking themselves blue over everything that went on and I don't have any additional, deep insights to add. Supposed "thought leaders" (like myself) have (publically and privately) questioned the status quo for quite a while, challenging the usefulness of EJBs, the practicality of separating the layers so profoundly, and identifying the needless complexity as a platform threatending problem (Rod has examples of major banks throwing away $20 million investments in Java and J2EE in favor of .Net). This philosophy has now gone completely mainstream. One has to question the relevance of EJB 3.0 at this time (and more so when it is ready for release). A phrase I came up with while talking to Rod is "Results Not Standards". Tapestry users are getting great results, even though Tapestry is not a standard (though it is compliant with the useful and reasonable parts of the J2EE standard). Likewise, WebWork, Spring, Hibernate (which is trying to become a standard) and so forth.

Tuesday, May 04, 2004

SDL, Testing, and the Scripting Debate

I'm really loving the effort I invested in SDL, it's very clean, very useful stuff. I'm beginning to productize Tapestry test framework and I'm targetting SDL, not XML, as the language for the scripts. Background: for Tapestry 3.0, I developed a "mock unit test" suite, more of an integration suite, where I simulate a servlet container around Tapestry pages and components. There's no HTTP involved, but none of the Tapestry objects know that ... they see the Servlet API and, for Tapestry's needs, it works just like the real thing.

Each script consists of a few definitions, and a series of sequential requests. Each request passes up query parameters, and makes assertions about the result of the request: mostly in terms of asserting text in the output (sometimes in the form of regular expressions).

However, the tests are pretty ugly; because a lot of the assertions are looking for HTML, there's lots of CDATA sections. The execution and parsing code is all twisted together and built on top of JDom. Line-precise error reporting came later, so it can be a challenge to find where, inside a test script, a failure occured. In addition, all the code is inside the junit folder, not part of the Tapestry framework itself, so it can only be used by Tapestry itself.

I'm currently starting to rebuild this support for use by Tapestry and for end-user applications. I'm building a better parser, using SDL as the script language, and building tests for the testing framework itself as I go (Who watches the Watchmen? Who tests the Testers?). Lots of work up front, but will easily pay for itself when we start adding more complicated tests for some of the 3.1 features ... I also expect it to run much faster.

Meanwhile, the debate about replacing XML and SDL in HiveMind with scripting rages on. I don't see the advantage to the scripting approaches ... they're all more verbose than the equivalent SDL. It's more code to do the same thing that you'd normally do by referencing builder factories. It won't document as HiveDoc, it raises many issues about multithreading. It adds unwanted dependencies to the HiveMind core framework. No one has made a compelling argument ... certainly not compelling enough for me to spend any time on that when I have other priorities, and so far, nobody else is checking code into the HiveMind CVS. So ... we have a fairly active HiveMind community but I'm still the only developer ... do I like this, or not?

Monday, May 03, 2004

Goodbye, Digester!

Ah, the evolution of XML parsing in Tapestry. Tapestry is very much driven by validated XML files (for page and component specifications, application specifications, and library specifications). In the earliest days, Tapestry was tied directly to Xerces. Later, it switched over to JAXP. I had reams of code that would walk the DOM tree and construct the specification objects from the XML.

As a nod to efficiency, I switched over in 3.0 to use Digester, but that's caused a lot of grief in its own right. It seems like the version Tapestry uses was always in conflict with whatever version was in use by the servlet container, especially Tomcat. That caused a lot of grief.

Meanwhile, Digester drags along some of its own dependencies, jakarta-collections and jakarta-beanutils. More JAR hell, keeping all those JARs and versions straight.

No more; I replaced Digester with an ad-hoc parser derived from (and sharing code with) the HiveMind module deployment descriptor parser. It uses a stack to track the objects being constructed (that's borrowed from Digester), but uses a simple case statement and some coding discipline, to deal with recognizing new elements and processing them. I haven't done any timings, but comparing this code to the Digester code leads me to think that this will have a substantial edge ... which will be even more important once Tapestry supports reloading of page templates and specifications. In addition, inside the monolithic SpecificationParser class, it's a lot clearer what's going on. The old code had to create some number (usually three, sometimes six) Digester rule objects for each Digester pattern (patterns are matched against elements on Digester's stack to determine which rules fire). The new code is almost entirely just private methods: beginState() methods that decide what state to enter based on the current element, enterState() methods that create new specification objects, push them onto the stack, and change to a new parser state, and endState() methods invoked when a close tag is found, to finalize created objects and pop them off the stack, and return the parser state to its earlier value.

Next up; the JDom-based parser for the Tapestry mock unit test suite. First, I want to use SDL, not XML, for these scripts. Second, I want them to run much, much faster and I suspect that a lot of time is being spent in JDom. Third, I want thrown assertion exceptions to have line-precise error reporting.

And fourth ... part of Tapestry 3.1 will be to productize this approach to testing Tapestry applications.

I'm the Seven of Clubs

On the just published Who's Who in Enterprise Java List (by The Middleware Company), I'm in the Clubs ("Pot Pourii") category, as the Seven of Clubs. If they do this again, I'd love to be recognized in the Hearts ("Contribution") category, along with Gavin King, Rod Johnson, Craig McClanahan and many others.

It's interesting just how many names on the list I don't recognize!

Someday, I'll have to find another picture of myself that I like ... this one was taken in 1999 atop Mt. Haleakala, Maui, Hawaii.

Sunday, May 02, 2004

Introduction to Jakarta Tapestry

And interesting link from Object Computing, Inc.: Introduction to Jakarta Tapestry.

The author, Rob Smith, is very positive on Tapestry ... he find's it fun! That should be the fifth goal of Tapestry (after simplicity, efficiency, consistency and feedback).