Friday, April 30, 2010

Inform 7's Birthday

Long, long ago, some of my first programs were interactive fictions. Really simple stuff, "go north", "kill rat", that kind of thing. While I was hacking that kind of thing together in Basic, others had gone much further: the masters of Infocom. This system, written way back in the late 70's and early 80's, predates Java but has many similar features; including a bytecode-based runtime portable across different operating system and hardware platforms.

Sure games have gone in a different direction with incredible 3d graphics, but there's still a certain joy in playing interactive fiction games; it really is like playing a short story. The games have evolved from classic dungeon crawls into something more, with many of the best games eschewing puzzles and focusing on interaction between the player and the non-player characters.

The modern way to write these games, if you are so inclined, is in the Inform 7 environment. The Inform team try to come up with new releases every year, on the "birthday"; this month is the 17th birthday of the original Inform language (Inform was originally a more C-like objected oriented language that's evolved over the years into its current state).

Here's a getting started screencast:

Inform 7 Introductory Screencast from Aaron Reed on Vimeo.

I've gone a bit deep with Inform in the past, and hope to do more in the future. It's a truly amazing piece of software ... the language is generally a natural language (I call it "the mother of all DSLs") with features combining object oriented, rules based and even aspect oriented programming. In fact, the next release, due shortly, even includes map and reduce operations! The language is very powerful, allowing for concise ways to deal with deep cross-cutting concerns, allowing for human ambiguity

More than that, the IDE is truly full and integrated. It has extensive documentation (both a front to back manual and a cookbook), and error messages include hyperlinks to your code and to the manual pages. It includes hundreds of short examples that can be pasted directly into the editor with a single click. It has built in testing features (shown in the screen cast above). It provides an incredible cross-reference of your project (integrated with the built-in libraries) ... even an automatic map of your game world. It's truly a labor of love, and I wish any of the tools I work with day to day showed so much innovation and usefulness.

If you've ever wanted to write interactive fiction, or just want to play with a really fascinating alternative language, give Inform a try.

Monday, April 19, 2010

Setting up committer access Git for Tapestry 5

Given the problems I'm having, I decided to set up a new local Git repository for futher work. Here's how to do it:

$ git clone git://git.apache.org/tapestry5.git

This sets up a new working folder, tapestry5. It takes it a while to download the necessary Git repository objects.

$ cd tapestry5
$ curl http://git.apache.org/authors.txt -o .git/authors.txt
$ git config svn.authorsfile .git/authors.txt

This fetches the current list of authors so that proper names appear in various Git reports, then configures Git to make use of the file.

$ git svn init --prefix=origin/ --trunk=trunk https://svn.apache.org/repos/asf/tapestry/tapestry5

This tells Git where to sync from and to.

$ git svn rebase

And that finishes things up, ensuring that you have all the most recent revisions.

From here on in, the two commands you need the most are git svn rebase (to pull in repository changes) and git svn dcommit (to push deltas back to Subversion). You should always rebase before a dcommit.

Perhaps that's not quite complete; I generally create local Git topic branches; so I start my work with git co -Blocal to create (or overwrite) my local branch, do my work there as a series of commits, then: git co trunk ; git rebase local to move those commits back over to trunk before git svn dcommit. This helps a lot if you ever have to deal with a merge.

When I'm fixing particular bugs, I often create a branch names after the bug id.

Update: Removed the --tags and --branches arguments ... mostly because of how horribly Git SVN works with branches (don't try it!), and to make the init step nice and fast.

Git & Svn : Not Always A Match Made In Heaven

Apache is stuck using Subversion ... so I've been using the Git/Svn integration built into Git for a while now. The good news is that most of the Git workflow comes with it ... you can create private branches, do local commits to your local repository, and build up a series of changes to dcommit ("delta commit") into SVN.

But that doesn't always work. For reasons I don't understand (given that there have been no other commits to SVN since I started work in my private branch), I keep getting the following error:

Applying: Provide the missing asset request handler for the virtual "stack" folder.
Committing to https://svn.apache.org/repos/asf/tapestry/tapestry5/trunk ...
 M tapestry-core/src/main/java/org/apache/tapestry5/internal/services/ClasspathAssetAliasManagerImpl.java
 M tapestry-core/src/main/java/org/apache/tapestry5/services/LibraryMapping.java
 M tapestry-core/src/test/java/org/apache/tapestry5/internal/services/ClasspathAssetAliasManagerImplTest.java
 A tapestry-core/src/test/java/org/apache/tapestry5/services/LibraryMappingTest.java
Committed r935569
W: 86533530aac8673a9e107e323de5201b7187270f and refs/remotes/origin/trunk differ, using rebase:
:040000 040000 9c78596ee3f916f012c51d8927b4aa31d497f17b 8eb2b9b4f28e825e223c736eaa664bb53018258e M tapestry-core
Current branch trunk is up to date.
# of revisions changed  
before:
 07b37e03cbc17012247d2221e795023c564d8228
0830b5f383dc94ae16088185efefac2e1358cf30
0bf378bcafc3f5372b67edc50d7de5bed8713cd0
95c87c5d7a2435df6bfced0d858bfdcb6ff26f22
3cd4ea4d9b225fd5013e1ce72cb9bac6d5b3e5e2
7b95d935099ebbbeb81845c1b8170a89d6ca6421
af310cfa9a5552aab2574c1e345b3beb049fb040
20805630fc67b83b4ca946b942716aeba4c80bef
b3ef5e069942a30e2dce45a35e4be16382c0108d
1be75a15b7f203c927bc2aa34f43dda59ca968e3
555d94ebab122a688b7a1c0af253bf73609f88f5
729757eb3c35e14e126cb6ef16f5032d95d1cc4a
79dcfa32b291454bf9c652d635374d60638b8fb8
304d12f9d7d040f4dc231d213df663fcdf3863b6
0d626a7b0648735ab83bc7a2fd241390eb92e4e2 

after:
 86533530aac8673a9e107e323de5201b7187270f
07b37e03cbc17012247d2221e795023c564d8228
0830b5f383dc94ae16088185efefac2e1358cf30
0bf378bcafc3f5372b67edc50d7de5bed8713cd0
95c87c5d7a2435df6bfced0d858bfdcb6ff26f22
3cd4ea4d9b225fd5013e1ce72cb9bac6d5b3e5e2
7b95d935099ebbbeb81845c1b8170a89d6ca6421
af310cfa9a5552aab2574c1e345b3beb049fb040
20805630fc67b83b4ca946b942716aeba4c80bef
b3ef5e069942a30e2dce45a35e4be16382c0108d
1be75a15b7f203c927bc2aa34f43dda59ca968e3
555d94ebab122a688b7a1c0af253bf73609f88f5
729757eb3c35e14e126cb6ef16f5032d95d1cc4a
79dcfa32b291454bf9c652d635374d60638b8fb8
304d12f9d7d040f4dc231d213df663fcdf3863b6
0d626a7b0648735ab83bc7a2fd241390eb92e4e2 
 If you are attempting to commit  merges, try running:
  git rebase --interactive --preserve-merges  refs/remotes/origin/trunk 
Before dcommitting
~/work/t5-project
$

I did the right things; git co trunk followed by git svn rebase, then git rebase revised-assets-12apr2010. It claimed to replay my branch changes on top of the trunk branch, but regardless, the dcommit failed.

Doing some hunting around with Google, I found a partial explanation, that at least gives me a way forward. I'd still like to know how I got into this predicament.

At this point I just keep blindly entering the command: git reset --hard 705ccfb1e27d303a9db62de755b2fcfcca9a02f6 ; git svn rebase; git svn dcommit and get one Git commit further each time (that's the Git hash code for my final change in my original branch). Joy.

Sunday, April 18, 2010

An extended stay in London

If you were unable to attend my "In the Brain Of" talk last Tuesday, it's now available at SkillsMatter ... voice, video and slides. This was a fun session, even if I was jet lagged to the point of being dizzy.

Meanwhile, I'm still hanging out in London until this at least the end of this week. I expect to work from my hotel room, visit a couple of clients, and maybe have a bit of fun. Drop me a line.

In London, looking for a Tapestry job? I've been contacted by a recruiter who is looking for you! They're building a team of perhaps 20 developers (I'm not yet sure who the actual client is). Drop me a line and I'll hook you up!

Tuesday, April 06, 2010

Meta-Programming Java with Tapestry

A significant amount of what Tapestry does is meta programming: code that modifies other code. Generally, we're talking about adding behavior to component classes, which are transformed as they are loaded into memory. The meta-programming is the code that sees all those annotations on methods and fields, and rebuilds the classes so that everything works at runtime.

Unlike AspectJ, Tapestry does all of its meta-programming at runtime. This fits in better with live class reloading, and also allows for loaded libraries to extend the meta-programming that's built-in to the framework.

All the facilities Tapestry has evolved to handle meta-programming make it easy to add new features. For example, I was doing some work with the Heartbeat enviromental object. Heartbeat allows you to schedule part of your behavior for "later". First off, why would you need this?

A simple example is the relationship between a Label component and a form control component such as TextField. In your template, you may use the two together:

  <t:label for="email"/>
  <t:textfield t:id="email"/>

The for parameter there is not a simple string, it is a component id. You can see that in the source for the Label component:

    @Parameter(name = "for", required = true, allowNull = false, defaultPrefix = BindingConstants.COMPONENT)
    private Field field;

Why does for="email" match agains the email component, and not some property of the page named email? That's what the defaultPrefix annotation attribute does: it says "pretend there's a component: prefix on the binding unless the programmer supplies an explicit prefix."

So you'd think that would wrap it up, we just need to do the following in the Label code:

  writer.element("label", "for", field.getClientId());

Right? Just ask the field for its client-side id and now all is happy.

Alas, that won't work. The Label component renders before the TextField, and the clientId property is not set until the TextField renders. What we need to do is wait until they've both rendered, and then fill in the for attribute after the fact.

That's where Heartbeat comes in. A Heartbeat represents a container such as a Loop or a Form. A Heartbeat starts, and accumulates deferred commands. When the Heartbeat ends, the deferred commands are executed. Also, Heartbeats can nest.

Using the Heartbeat, we can wait until the end of the current heartbeat after both the Label and the TextField have rendered and then get an accurate view of the field's client-side id. Since Tapestry renders a DOM (not a simple text stream) we can modify the Label's DOM Element after the fact.

Without the meta-programming, it looks like this:

    @Environmental
    private Heartbeat heartbeat;

    private Element labelElement;

    boolean beginRender(MarkupWriter writer)
    {
        final Field field = this.field;

        decorator.beforeLabel(field);

        labelElement = writer.element("label");

        resources.renderInformalParameters(writer);

        Runnable command = new Runnable()
        {
            public void run()
            {
                String fieldId = field.getClientId();

                labelElement.forceAttributes("for", fieldId, "id", fieldId + "-label");

                decorator.insideLabel(field, labelElement);          
            }
        };
        
        heartbeat.defer(command);

        return !ignoreBody;
    }

See, we've gotten the active Heartbeat instance for this request and we provide a command, as a Runnable. We capture the label's Element in an instance variable, and force the values of the for (and id) attributes. Notice all the steps: inject the Heartbeat environmental, create the Runnable, and pass it to defer().

So where does the meta-programming come in? Well, since Java doesn't have closures, it has a pattern of using component methods for the same function. Following that line of reasoning, we can replace the Runnable instance with a method call that has special semantics, triggered by an annotation:

    private Element labelElement;

    boolean beginRender(MarkupWriter writer)
    {
        final Field field = this.field;

        decorator.beforeLabel(field);

        labelElement = writer.element("label");

        resources.renderInformalParameters(writer);

        updateAttributes();

        return !ignoreBody;
    }

    @HeartbeatDeferred
    private void updateAttributes()
    {
        String fieldId = field.getClientId();

        labelElement.forceAttributes("for", fieldId, "id", fieldId + "-label");

        decorator.insideLabel(field, labelElement);
    }

See what's gone on here? We invoke updateAttributes, but because of this new annotation, @HeartbeatDeferred, the code doesn't execute immediately, it waits for the end of the current heartbeat.

What's more surprising is how little code is necessary to accomplish this. First, the new annotation:

@Target(ElementType.METHOD)
@Retention(RUNTIME)
@Documented
@UseWith(
{ COMPONENT, MIXIN, PAGE })
public @interface HeartbeatDeferred
{

}

The @UseWith annotation is for documentation purposes only, to make it clear that this annotation is for use with components, pages and mixins ... but can't be expected to work elsewhere, such as in services layer objects.

Next we need the actual meta-programming code. Component meta-programming is accomplished by classes that implement the ComponentClassTransformationWorker interface.

public class HeartbeatDeferredWorker implements ComponentClassTransformWorker
{
  private final Heartbeat heartbeat;

  private final ComponentMethodAdvice deferredAdvice = new ComponentMethodAdvice()
  {
    public void advise(final ComponentMethodInvocation invocation)
    {
      heartbeat.defer(new Runnable()
      {

        public void run()
        {
          invocation.proceed();
        }
      });
    }
  };

  public HeartbeatDeferredWorker(Heartbeat heartbeat)
  {
    this.heartbeat = heartbeat;
  }

  public void transform(ClassTransformation transformation, MutableComponentModel model)
  {
    for (TransformMethod method : transformation.matchMethodsWithAnnotation(HeartbeatDeferred.class))
    {
      deferMethodInvocations(method);
    }
  }

  void deferMethodInvocations(TransformMethod method)
  {
    validateVoid(method);

    validateNoCheckedExceptions(method);

    method.addAdvice(deferredAdvice);

  }

  private void validateNoCheckedExceptions(TransformMethod method)
  {
    if (method.getSignature().getExceptionTypes().length > 0)
      throw new RuntimeException(
          String
              .format(
                  "Method %s is not compatible with the @HeartbeatDeferred annotation, as it throws checked exceptions.",
                  method.getMethodIdentifier()));
  }

  private void validateVoid(TransformMethod method)
  {
    if (!method.getSignature().getReturnType().equals("void"))
      throw new RuntimeException(String.format(
          "Method %s is not compatible with the @HeartbeatDeferred annotation, as it is not a void method.",
          method.getMethodIdentifier()));
  }
}

It all comes down to method advice. We can provide method advice that executes around the call to the annotated method.

When advice is triggered, it does not call invocation.proceed() immediately, to continue on to the original method. Instead, it builds a Runnable command that it defers into the Heartbeat. When the command is executed, the invocation finally does proceed and the annotated method finally gets invoked.

That just leaves a bit of configuration code to wire this up. Tapestry uses a chain-of-command to identify all the different workers (theres more than a dozen built in) that get their chance to transform component classes. Since HeartbeatDeferredWorker is part of Tapestry, we need to extend contributeComponentClassTransformWorker() in TapestryModule:

  public static void contributeComponentClassTransformWorker(
      OrderedConfiguration<ComponentClassTransformWorker> configuration
  {
  
    ...
    
    configuration.addInstance("HeartbeatDeferred", HeartbeatDeferredWorker.class, "after:RenderPhase");
  }      

Meta-programming gives you the ability to change the semantics of Java programs and eliminate boiler-plate code while you're at it. Because Tapestry is a managed environment (it loads, transforms and instantiates the component classes) it is a great platform for meta-programming. Whether your concerns are security, caching, monitoring, parallelization or something else entirely, Tapestry gives you the facilities to you need to move Java from what it is to what you would like it to be.