Saturday, March 26, 2011

Combining Gradle with Antlr3

I've been going through a relatively painless process of converting Tapestry from Maven to Gradle, and am thrilled with the results. My biggest stumbling point so far was Tapestry's use of Antlr3 for its property expression language.

The built-in support for Antlr only went as far as Antlr2. The Maven plugin I had been using understood Antlr3. After a bit of research and hacking, this is what I came up with as a solution for Tapestry:

description="Central module for Tapestry, containing all core services and components"

antlrSource = "src/main/antlr"
antlrOutput = "$buildDir/generated-sources/antlr"

configurations {
  antlr3
} 

sourceSets.main.java.srcDir antlrOutput

dependencies {
  compile project(':tapestry-ioc')
  compile project(':tapestry-json')
  
  provided project(":tapestry-test")
  provided "javax.servlet:servlet-api:$servletAPIVersion"

  compile "commons-codec:commons-codec:1.3"

  // Transitive will bring in the unwanted string template library as well
  compile "org.antlr:antlr-runtime:3.3", { transitive = false }

  // Antlr3 tool path used with the antlr3 task
  antlr3 "org.antlr:antlr:3.3"
}

// This may spin out as a plugin once we've got the details down pat

task generateGrammarSource {
  description = "Generates Java sources from Antlr3 grammars."
  inputs.dir file(antlrSource)
  outputs.dir file(antlrOutput)
} << {
  mkdir(antlrOutput)
  
  // Might have a problem here if the current directory has a space in its name
  
  def grammars = fileTree(antlrSource).include("**/*.g")
    
  ant.java(classname: 'org.antlr.Tool', fork: true, classpath: "${configurations.antlr3.asPath}") {
     arg(line: "-o ${antlrOutput}/org/apache/tapestry5/internal/antlr")
     arg(line: grammars.files.join(" "))
  }
}

compileJava.dependsOn generateGrammarSource

The essence here is to create a configuration (a kind of class path) just for running the Antlr Tool class. The new task finds the grammar files and feeds them to the tool. We also thread the output of the tool as a search path for the main Java compilation task. Finally, we define the inputs and outputs for the task, so that Gradle can decide whether it is necessary to even run the task.

Part of the fun of Gradle is that it is still a Groovy script, so there's a familiar and uniform syntax to defining variables and doing other non-declarative things, such as building up the list of grammar files for the Tool.

As you might guess from some of the comments, this is something of a first pass; the Maven plugin was a bit better at assembling the list of input file names in such a way that the Antlr3 Tool class knew where to write the output Java source files properly; if Tapestry used a number of grammars in a number of different locations, the solution above would be insufficient. It also seems roundabout to use Ant to launch a Java application ... I didn't see an easier way (though I have no doubt its hidden inside the Gradle documentation).

My experience getting this working was mostly positive; there's a very large amount of documentation for Gradle that helped, though it can be a bit daunting, as the information you need is often scattered across a mix of the Gradle DSL reference, the User Guide, the Javadoc and the GroovyDoc. Too often, it feels like a solution is only understandable once finished, working backwards from some internal details of Gradle (such as which exact classes it chooses to instantiate in a given situation) back through the various interfaces, Java classes, and Groovy MetaObject extensions to those classes.

In fact, key parts of what I did ultimately accomplish were discovered through web searches, not in the documentation. But, that also means that the system works.

Of course, this is the pot calling the kettle black ... one criticism of Tapestry can be paraphrased as we can customize it to do anything, and in just a few lines of code, but it can take three days to figure out where those lines of code go.

At the end of the day, I'm much happier with Gradle; the build process is faster, the build scripts are tiny and much, much easier to maintain, and the feedback from the tool is excellent. There's still many more issues to work out ... mostly in terms of Apache and Maven infrastructure:

  • Ensuring the Maven artifacts are created properly, with the right dependencies in the generated pom.xml
  • Generating a Maven archetype using Gradle
  • Generating JavaDoc and Tapestry component documentation with Gradle, along with a minimal amount of pages to link it together (akin to the Maven site plugin)
  • Generating source and binary artifacts and getting everything uploaded to the Apache Nexus properly

Regardless, I think all of these things will come together in good time. I'm not going back, and dearly hope to never use Maven again!

3 comments:

  1. Nice! Thanks for sharing.

    Some ideas for minor tweaks:

    * I think you may be able to get away without converting configurations.antlr3.asPath to a String.
    * You can use the JavaExec task to eliminate the doLast block of your task (although you might still need a doFirst for the mkdir still).
    * To address spaces in the path you should be able to change grammars.files.join(" ")) to grammars.files..collect { '"'+it+'"' }.join(" "))
    * You could create a separate sourceSet for the generated code. i.e. sourceSets.generated.java.srcDir = ...
    * If you wanted you could inline the antlrOutput and antlrSource properties and read the antlrOutput value from sourceSets.generated.java.srcDir.

    ReplyDelete
  2. Further to what the Bunny said, you should definitely make generateGrammarSource a JavaExec task and then simply have it depend on a Directory task created using the dir(...) shorthand to generate the antlrOutput directory.

    ReplyDelete
  3. I was responsible for the Antlr plugin. I had only done Antlr 2 because, well that is all I needed for Hibernate. We are looking at moving Hibernate to Antlr 3 and in fact have already started the work a bit. But that is all based on code from the maven-based builds. The good news is that I had implemented the Antlr3 support and support for GUnit testing in the maven Antlr plugin, so porting that to Gradle will be minor.

    ReplyDelete

Please note that this is not a support forum for Tapestry. Requests for help will be deleted. Please subscribe to the Tapestry user mailing list if you are in need of support, or contact me directly for professional (for pay) support.

Spammers: Don't bother. I delete your comments and it's a waste of time for both of us. 垃圾邮件发送者:不要打扰。我删除您的评论和它的时间对我们双方的浪费