[Off Topic] Roller Coaster Tycoon 3 demo

I downloaded the Roller Coaster Tycoon 3 demo this morning. I've always been a big fan of the original Roller Coaster Tycoon (RC1), even though I never came close to finishing it. I just like building the parks and letting the little people run through it.

I've always thought it was marvelous that a game that was fully creative, non-violent (well, except for dropping the peeps in the pond) and intricate was so popular.

It looks like they've retained the essential nature of the original. You are still building your park on a grid, but your eye, your camera, is free to roam much more freely ... rotating and zooming to your heart's content.

The ability to get a peep's eye view of any ride, not just roller coasters, is wonderful.

The demo is a major power hog; my laptop (A Dell Inspiron 8200, 1 gig of ram, 2ghz P4, GeForce 4 440 Go) is somewhat taxed ... it will run full screen at 800x600 but only just. I suspect larger parks will be a problem. This game is going to cost me a lot of cash for a new system.

I haven't found where you get to review all your peep's thoughts (I hope that's still in there). They've also made the concession stands overly complex (you get to define a whole menu, control how many pickles are put on each burger and a bunch of other stuff nobody will ever do).

The biggest win is that you can pause the game to do construction. That was always my biggest beef with RC1 ... constant interruptions when building your big coaster.

The peeps are much more involved; different sexes and ages. There's some kind of group (or family) system.

Parks also simulate day and night as well as season; the park looks terrific at night, and there are fireworks shows!

Alas, back to work for now!

Feedback++: Going beyond Line Precise Error Reporting

This I like ... for Laurent Etiemble, even line precise error reporting is not enough, so he has made some improvements to display the actual text that is in error. Details in his blog. It looks like:

This is certainly something that could show up in Tapestry 3.1 (either by duplicating Laurent's work, or getting him to properly donate it). Additionally, Geoff has talked about some kind of hook that would make the stack trace clickable links ... clicking the links would open the correct file inside Eclipse.

HiveMind 1.0 final tomorrow!

The votes have been cast and, predictably, HiveMind 1.0 final release has been approved. The web site has been updated, an the Maven repository has been seeded with the 1.0 final jars. The distributions have been uploaded to Apache, and the only thing left is to wait about 24 hours for the Apache mirrors to pick up the files. Tomorrow morning I'll update the Jakarta web site and submit the news item to TheServerSide.

And still, so much more to do in HiveMind 1.1 and in Tapestry 3.1.

Tapestry 3.1 and backwards compatibility

I've gotten a couple of concerned notes about backwards compatibility from Tapestry 3.1 back to Tapestry 3.0. People are concerned they may have to throw away some or all of their 3.0 applications when they upgrade to 3.1.

Tapestry has historically had a fuzzy distinction between APIs (application program interfaces, intended for exposure to, and use by, end-user developers) and SPIs (service provider interfaces, or at the very least, internal interfaces that are expected to change between releases). The interfaces in the org.apache.tapestry package are generally APIs , and the interfaces and classes is other packages are more often SPIs.

Despite the large amount of refactoring so far, nothing I'd consider an API has changed, though some SPIs have changed.

I'm working to make the upgrade path from 3.0 to 3.1 pretty painless for 90%+ of users.

• It is possible that the <extension> element will no longer be honored (you'll get a runtime warning).
• It is likely that the <service> element will no longer be honored.
• It is possible that it will no longer be allowed to sub-class BaseEngine. Certainly the AbstractEngine and BaseEngine classes have changed significantly, and will likely merge together.
• Parameter directions from 3.0 DTD specs will be ignored, and will always be implemented using the new smart caching connected properties code (yet to be written).

I think it will be a reasonable upgrade path from 3.0 to 3.1. For simple applications and pages, nothing will be necessary. For a minority of applications, there will be some localized changes.

It would be very nice to more cleanly separate internal interfaces and classes from the public APIs. However, doing this properly will certainly break backwards compatibility.

Part of the problem is the use of inheritance in Tapestry. In Tapestry 3.x, your pages and components must subclass from Tapestry base classes. This exposes much internal implementation details about those classes, dragging things that should be firmly in the "internals" category out into the open.

Part of my vision for Tapestry 4.0 (besides stripping the leading "I" off any remaining interfaces) is to break the inheritance requirement. In that release, your pages and components will be true POJOs. This will be an even better separation than is possible today. Something very much like todays IPage/IComponent hierarchy will exist .. but that will be largely be internal to Tapestry. The classes you write will be peers of the actual components. In general, your code will not access the component directly, but will instead have information, properties and services from the component injected into your classes.

I'm very excited by this potential, but first things first ... have to get 3.1 up and running and not leave all the 3.0 users behind!

Whew! Full speed ahead!

The vast refactoring of Tapestry continues. Today I completed my first pass. All the major subsystems of Tapestry (such as ISpecificationSource, ITemplateSource, DataSqueezer, IScriptSource, etc.) are now HiveMind services. Yes ... everything still runs pretty much identically, but under the covers the code is evolving into smaller, simpler classes.

In addition, the first real changes (still, not visible though) are now in place. There is now a collection of BindingFactory implementations, one for each type of binding (OGNL expression, message keys, and literals). There's a BindingSource service that knows how to take a string (from an HTML attribute value) and find the correct BindingFactory, based on prefix, to generate a IBinding instance from it.

This is now being used for all bindings created by the SpecificationSource and the ComponentTemplateLoader. Its driven by a configuration, which means that it will be super easy to add additional binding prefixes (even application specific ones). This is the first, approachable, step along the path to what will be Tapestry 3.1.

It's a great, iterative process. New code I create is truly testable ... I hope to start retiring many of the integration tests (the ones driven by XML script files) since those are pretty darn slow to execute compared to focused unit tests.

I had one great headache; I converted a utility class (BaseComponentTemplateLoader) into a service (ComponentTemplateLoader). BaseComponent used to create one instance of this every time a component loaded its template. I converted this to a threaded service, because it had some internal state.

After many headaches, I realized that BaseComponent was doing it right ... the state wasn't merely threaded, it was reentrant. Loading a component will often cause a new component to be created ... and if that new component has a template, the process goes recursive. I didn't realize that, and started getting bizarre, impossible results. Eventually, I realized what was going on ... and my final solution is very similar to where I started. ComponentTemplateLoaderImpl is a normal service (not threaded), but delegates just about all of its behavior to ComponentTemplateLoaderLogic, which is created (as before) for each component and discarded immediately.

This makes me think that HiveMind needs a reentrant service model. I think Spring has something like this, "prototype beans", where each method invocation causes a new implementation instance to be created. That would be easy enough to do in HiveMind.

It has been very slick seeing the mix of service implementations; most of the Tapestry services are ordinary singletons. A few, which hold client-specific state for the duration of a request, are threaded. However, when these are combined together, the singletons and threaded code doesn't have to know anything when they invoke each other ... its just objects and interfaces and the mechanics of locating a thread-specific instance to process a particular service method is entirely hidden.

So, Tapestry is getting more powerful and simpler at the same time. That's hard to beat. Not sure when I'll have time for more or what I'll hit next ... I'm following a kind of round-robin approach, where even the code I've created recently is ready to be refactored into smaller and smaller pieces. Probably next up is to start thinking about engine services. There's also refactoring around OGNL 3.0. Or getting ready for the major rework for modularity (splitting the application across multiple folders). It's all good!

[Off Topic] VoteOrNot.org

Everyone knows its important to vote. The stakes are extraordinarily high for the current presidential election ... but even so, voter turnout in this country is abysmally low.

I'm proud to say that I've voted in every presidential election since I was 18, including most primaries. I've also voted in every (I think) state-wide election. Too many people, though, let themselves get sidetracked.

How about if someone offered you $100,000 to vote? Oh, and $100,000 to a friend who got you to vote? That's what VoteOrNot.org is doing. Go to their site, promise to vote, and possibly win after the election.

NEJUG - Erik Hatcher on Lucene

Erik Hatcher will be in town next week to present on the Lucene text searching/indexing tool to the NEJUG (North East Java User's Group). This is occuring 5:30pm on September 16th. If you can snag a reservation, you should ... Erik's a great presenter and Lucene is a great (and largely unrecognized) technology.

Progress on all fronts

Tapestry 3.1 is really getting into gear now; I've been working my way forward from the servlet, converting everything into HiveMind services, pipelines and configurations. I've also said sayonara to revision histories and been moving source code into "standard" directories (that is, src/java). I can see switching to Subversion for source control someday, just because it does a proper job of moves and renames -- history on files is maintained even when the file is renamed or moved to a new directory. It seems like CVS is pretty antagonistic towards refactoring, especially in context of the XP mantra of fearless, merciless, constant refactoring.

Tapestry's bug list is now under JIRA, which is so much easier, faster and more sensible than Bugzilla.

I've taken a pause from working on the Tapestry code to work on the Tapestry documentation. The awkward DocBook stuff is being phased out ... all the documentation will be converted to Forrest format, which means better and more consistent navigation. I realized that unless I had the Tapestry 3.0 documentation in a ready to edit form, I would not update it as I was making the real changes in the 3.1 code. As I found during Tapestry 3.0's too-long development cycle, if you don't keep the docs up to date as you work, your problems just multiply. With HiveMind, I've worked diligently to keep docs up to date at all times, and I want to start on the right footing for Tapestry 3.1 as well.

James Carman looks likely to be the newest HiveMind committer, the vote is in progress but opposition is unlikely. Meanwhile, I've just fixed a few nasty little bugs in HiveMind, in prep for release candidate 2. Nasty, of course, always means class loaders and/or thread concurrency. Now, throw dynamic class fabrication into the mix.

And I signed a contract for three months of work, possibly more (details to come). So the good news is that I have money coming in and am doing Tapestry work full time. The bad news is I don't have huge amounts of time to do things like build out the Tapestry lab course I've been dreaming of, or do the Hibernate research I've been hankering for (until and unless that becomes part of my contract). I've actually been full time with this client for almost two months; I've been letting others pick up the slack in terms of answering questions on the mailing lists while I do more and more heavy lifting on the code front (especially on the Tapestry side).

Much more urgent is the need to get out of the house and hit the beach! Summer's ending and I'm still pretty pasty-white. Time to break out the boogie boards and hit the waves.

ScrapeSDL -- Convert SDL back to XML

With the release of HiveMind 1.0-rc-1, I've managed to "strand" a number of HiveMind users who had adopted SDL (Simple Data Language). SDL is no longer supported.

ScapeSDL.zip is an Eclipse project containing code that can convert an SDL file into an XML file. It's what I used to convert all the SDL in HiveMind itself back into XML (though I did have to do a bit of manual reformatting).

Tapestry and HiveMind at ApacheCon

Last year, I did a short, 60 minute, session on Tapestry at ApacheCon ... and I really think that launched things for me. I got a lot of good feedback on Tapestry and on my speaking style, and its opened a number of doors for me.

This year at ApacheCon, I'll be doing a long session on Tapestry on Sunday that will cover most of the material in my two normal Tapestry sessions (form building and component building). Additionally, I'll have a HiveMind session late on Wednesday. Details may someday be posted on the ApacheCon schedule.

Erik Hatcher, Matt Raible, and many others will be there. It was a good time last year (of course, I did bring my wife, Suzanne).

I'm spending too much time in Vegas, for someone who doesn't gamble. I was there in April for TheServerSide Symposium, I'll be there in October for a friend's birthday, and back in November for ApacheCon. Thankfully, I have my rolling laptop case (what a difference that makes when your laptop weighs in at about 15 pounds!).

HiveMind 1.0-rc-1

Following a successful vote (notable only for the absence of a vote from Knut), I'm building out HiveMind 1.0-rc-1 at this very moment. That's Release Candidate 1 ... and I expect to be rolling to a final 1.0 release very quickly.

There are a boat-load of plans for 1.1 and I hope that, unlike Tapestry's lethargic pace, we can get a 1.1 out in just a matter of months or even weeks.

"Invite 6 friends to GMail" -- It's open season!

I've built up six GMail invites ... drop me an email at hlship AT gmail DOT com if you'd like one. First come, first served.

Update: Sorry, all gone!

Dependency Injection -- the mirror of Garbage Collection

A ways back, the idea of a garbage collected language was tantamount to heresy. "It'll be slow." "It isn't necessary." "How hard is malloc()?" "Reference counting is all you need." etc.

Nowadays, garbage collection is the norm and there's no going back. By freeing us of concerns about allocating and freeing memory, new development techniques became possible. For me, personally, it was very liberating ... I've always coded in terms of small modules working together. Fifteen years ago I was writing almost-object-oriented code in a heavy duty procedural language (PL/1) ... it was just much, much harder.

Garbage collection liberates me to solve problems in my way ... using lots of small, well understood objects working together. When you have to be concerned about every object creation and destruction, you get miserly about them and before you know it, you have these big, monolithic objects that break all the now accepted rules about separation of concerns. You saw this inside the otherwise stellar NeXTSTEP libraries ... there were quite a few kitchen-sink objects and your main extension point was to subclass an existing class.

I used to be dismayed that the UML sequence diagrams did a really poor job of illustrating inheritance. It was really hard to properly diagram an object invoking a method in itself that's implemented in a super-class. Nowadays, I realize that the Amigos were ahead of the curve: aggregation trumps inheritance. And don't think I haven't been thinking about this in terms of Tapestry, which mandates inheritance (you must subclass AbstractComponent, BaseComponent or BasePage). I have been, and some future release of Tapestry will break that requirement.

But the story doesn't end with garbage collection (even if the individual objects do). Garbage collection is the last stage of an object's life cycle, but there's just as much going on at the start of the object's life cycle. That's why component frameworks and dependency injection containers (such as HiveMind, Spring, Picocontainer and Avalon) are so important.

"Why do we need an external descriptor?" "It just isn't necessary." "How hard is it to new Foo()?" Sound familiar?

Once you start thinking in terms of large numbers of objects, and a whole lot of just-in-time object creation and configuration, the question of how to create a new object doesn't change (that's what new is for) ... but the questions when and who become difficult to tackle. Especially when the when is very dynamic, due to just-in-time instantiation, and the who is unknown, because there are so many places a particular object may be used.

HiveMind didn't spring up out of thin air, it was based on me seeing an awful lot of code, production code, that was either a) inefficient, b) unnecessary or c) incorrect (often due to errors related to class loaders and multi threading). Too often it has been option d) all of the above.

Dependency injection acknowledges an initial state for objects ... the construction state, where objects are created and configured before being put into production. With HiveMind, you get the benefits of simplicity: your services are fundamentally POJOs. But you also get all the life cycle benefits: just-in-time, thread safe construction. Any reasonably sized system is going to need those benefits ... they are a cornerstone of providing the efficiency and robustness your clients and customers require. If you don't get them from a container such as HiveMind, you'll be writing that code yourself. Again and again, introducing bugs all the way.

You're willing to let something else manage the death of your objects because it, the Garbage Collector, can do a better job than you can. Likewise, you should accept that a container, whose only responsibility is to construct and configure your objects, will do a better job of it than your own code. Embrace that fact ... and get back to interesting work!

Groovy and Tapestry: Followup

As a follow up to my earlier post about combining Tapestry and Groovy, the folks involved (Michael Henderson and Richard Hensley) have combined their implementations and set up a SourceForge project.

It looks pretty sweet; you create a script and static methods in the script can be listener methods. Also, by using the correct method names (say, pageBeginRender), you script's static methods will be invoked at the right times. Also, static listener methods.

I think Hani had a diatribe not long about about flexibility. However, one of the core features of Tapestry is how it is divided into subsystems, precisely to provide this kind of flexibility ... such as changing the very nature of the framework! For the vast majority of users, that flexibility is masked as unnecessary complexity (which is probably the root of Hani's blog-arrhea). But often, that complexity is a reflection of a far reaching vision. Tapestry has always supported the notion that some pages or components would not be files in the WAR, but would be located from an external source, or even created dynamically at runtime. The bridge code that allows Groovy scripts to act like pages is one offshoot of that vision.

Atlanta Java User's Group Appearance

I'll be demonstrating Tapestry and Spindle on Aug. 17th at the Atlanta Java User's Group. Thanks to Jay Zimmerman of No Fluff Just Stuff for setting it up!

HiveMind happenings

So, by developer vote, SDL (Simple Data Language) has been removed, and the ocean may return to a gentle simmer. I learned a bit about JavaCC with SDL and was excited by it, but I'd rather concentrate on what HiveMind does really well, and not be sidetracked. Instead, we're looking for ways to further reduce the size and complexity of HiveMind's XML module deployment descriptors, and reduce the amount of code in general.

One thing else I've been working on is the ServicePropertyFactory. The idea here is that sometimes service A needs a property from service B to do its work. One approach would be to use service-property:B:foo to inject property foo from service B into service A.

But what if the value can change over time? In Tapestry, examples of this will be access to the HttpServletRequest, HttpServletResponse and HttpSession objects for the current user and current thread. These objects are specific to one thread and change on every request.

Your code could hold a reference to service B and get the property from B every time it is needed. But that's more code, more code paths, more Law of Demeter violations.

What would be nice would be if that property of B could be bound to service A? That's what ServicePropertyFactory does. It creates a service implementation, a proxy. The proxy keeps a reference to service B and, on each method invocation, gets the property from B and re-invokes the method on the property. This is service C, the binding between A and B, and is what gets injected into A.

This allows the code for A to treat a property (say, the HttpServletRequest) as if it was unchanging, when in fact, it is constantly changing. There are pitfalls ... the cost of accessing the methods of the proxy is higher than an ordinary object (it has to go get the real value from the underlying service), and you wouldn't want service A to pull any information out of the property and store it, since that kind of data will likely not be valid after the current request.

What this does do is allow the more dynamic parts of the HiveMind application to be isolated in their own services and allows the rest of the code to stay simple.

Ant best practice: One Compile, One Directory, One JAR

Do yourself a favor. When you are laying out your source tree, follow this rule: One Compile, One Directory, One JAR. Just because Ant supports includes and excludes on its <javac> and <jar> tasks doesn't mean you should actually use them!

A proper source tree structure may have multiple source code roots, and will run javac multiple times -- but each execution will compile all files under a particular source directory into its own, individual classes directory. The contents of that directory (perhaps combined with some deployment descriptors or other resources) will form a JAR. If anything depends on that code, it should add the JAR (not the classes directory, and certainly not the source directory) to its compile time classpath.

Why is the compile-and-split approach dangerous? It makes the build process and build scripts more complicated -- possibly lethally so. It ensures that the build process will not generate class files the same way (and into the same directories) as the IDE. It increases the likelyhood that some classes will be duplicated (compiled multiple times to multiple directories and packaged into multiple JARs), which can easily lead to mysterious ClassCastExceptions at runtime.

A particularly important case involves deployment into an application server. Once you get into complex EAR deployments (with multiple EJB JARs and WARs), the class loader issues can quickly get out of control. By using multiple compiles, you can cleanly ensure that the libraries available at build time match the libraries available to the runtime classloader for that artifact (that EJB JAR or WAR). Without this simple bit of discipline and organization, you can lose big chunks of time to tracking down ClassNotFoundExceptions.

I'm always amazed when I see this kind of thing in otherwise sane code and build environments. I've also seen just how far wrong this approach can go ... such as one source tree, one compile, 40+ JAR files, 10,000+ lines of build scripts!!

.Net vs. Struts ... and the winner is Tapestry!

Just something I noticed on the Cardsharp on Software blog:

Tapestry from the Jakarta Project. Yes, it's not JSF. Yes it's "non-standard." Yes, it requires you to learn a new library and a new way of doing things. But it's better. Far less configuration than Struts, and what is there makes sense. Instead of a jumble of disconnected concepts, the whole library is built around an event-driven mental model that just makes sense. Things that would take me 5 lines of code in Struts take one or two in Tapestry. Imagine being able to tie an event handler in a Java class to a link or a button with no configuration file changes! Think about forwarding to a new page and passing objects to the "action" by setting properties on it istead of hucking everthing into the request or session. Think about how much you like Tiles and imagine what it would be like if everything was a tile, but didn't require all the verbose <tiles:put> lines just to set parameters on your tiles. Think about all those times you'd wished you could put custom configuration information into your <ACTION> tags in the struts-config.xml (especially if you've tried to customize it using the <set-property> nonsense!). Think about how nice your life would be if your ActionForm and your Action were actually in the same class in a way that made sense, and think about how nice it'd be if the framework implemented all the getters and setters for you. Think about implementing complex interactions without jumping through all kinds of hoops to manage the state of your page.

And I'm only scratching the surface here.

This Great Hacker Does Use Java

Contrary to Paul Graham's misguided treatise on Great Hackers, this great hacker (by his definition, I prefer the term "developer") absolutely does use Java and loves it. You can pry it out of my cold, dead hands kind of love.

Side note: coming from an Objective-C background, I was initially very dismissive myself of Java. I still prefer many aspects of the Objective-C syntax, and Objective-C has killer runtime libraries ... but I've grown to love and be reliant upon Java type safety, and garbage collection. Both are pre-requisites for solving problems in an object oriented way: using large numbers of small, simple objects.

I feel there is a kind of insipid game out there I call "chase the gigahertz". Every time the language (COBOL, then C, then C++, then Java, now Ruby and friends) catches up to the available computing power and the available libraries are in a state where you can actually get something accomplished, there's a lemming-like urge to jump ship and start all over again in the latest and greatest ... with a less efficient, less optimized environment that pushes the performance barrier back up and out of reach.

Wait. Stop. Ask yourself "why?". As the differences between generations of languages become more and more subtle, you have to ask yourself ... what do I gain by switching? What cost do I pay? Am I throwing out the Baby with the Bathwater? Sure, you'll be incrementally more effective at doing the kind of things that look good in demos and tutorials ... but the moment you reach for your handy XML parser and it isn't there, or a friendly GUI toolkit and you have to start from scratch. Database access. Command line parsing. Transaction management. Yes, analogues exist for all of these, but you'll be starting from scratch to learn their APIS, quirks, bugs and workarounds. Just how compelling is that nifty bit of syntax, those couple of characters you don't have to type, that convenient built-in function?

For me, Java hits a terrific sweet spot. Vast numbers of open-source libraries. Very competent language. Minimal amounts of stupid syntax. Of course, there are things I'd change ... but in my analysis, its not about the language syntax its about how you use the language. I'm a framework guy; when I get annoyed at writing dumb, repetitive code I build a framework to take care of that dumb, repetitive stuff for me. I write it once, test it, forget how it works. Tapestry saves me a lot of trouble when dealing with the ugly intricacies of web applications. HiveMind saves me a lot of trouble when building complex apps from lots of simple little pieces. I don't throw out years of effort and hope some new language will solve my problems for me ... I face up to my issues and fix them right here, right now, in Java and be done with it.

Java is good enough. Sometimes "good enough" today is better than "ideal" tomorrow. I can get insanely great things done in Java today, and tomorrow, and the day after.

I'm not saying you should be complacent. Dave Thomas recommends learning a new language at least every year, to give you more experience, more insight into new ways of accomplishing things in your language of choice. I hope the Java language designers to the same, and give us some stuff we could use (such as continuations and closures). As a software craftsman, your technique should transcend any one programming language.

But back to Paul Graham; he's making that tired claim about productivity, as if productivity is linked to characters typed. So typing productivity trumps everything else? As you can see, I don't think so. I think he's fallen prey to a bad analogy ... that because some great hackers chose to use scripting languages such as Perl, Python and Ruby then all productive developers should use those languages for all chores, great and small. He's decided that one language (or class of language) is universally correct, and he's using that as a litmus test for being a great hacker.

Whereas, I think "the appropriate tool for the job", scripting languages for simple, scripting things. Power languages, such as Java, for long term, high quality, maintainable development. Individuals can move mountains, if they have the right tools. For the applications and frameworks I build, the right tools are Java, the Java runtime and the innumerable Java libraries.

German Tapestry Book Update

As mentioned previously, a German language book on Tapestry has been published ... and I now have a copy of it myself (thanks, Stefan!). Sorry, no review ... I don't read German, which means I can't even read my own vorwort (forward).