Tapestry Training -- From The Source

Let me help you get your team up to speed in Tapestry ... fast. Visit howardlewisship.com for details on training, mentoring and support!

Thursday, February 05, 2009

A Better Web Framework: Tapestry's Response

Last month, Ibrahim Levent published a detailed posting about the capabilities of a better web framework on his blog.

I'd thought I'd describe my reaction to it, as well as how Tapestry today, and Tapestry in the future, fits in with this vision.

Certainly there's a lot here, and there's a troubling lack of focus: Mr. Levent is demanding very specific features that span a number of domains. In effect, he's asking for an application server vendor to deliver the One True Stack ... to which I say "good luck with that!"

1- Includes all core application layers (MVC):

Web framework should include data access, business logic, controller and presentation layers internally. As frameworks turn out to be an integration hub, it looses value. Every integration among the core layers introduces new complexity, new glue code, new dependency, and conflicting of intersecting features. If data access layer (Model) uses another framework, presentation layer (View) uses another framework, integrating these frameworks adds a very big challenge even if frameworks support each other. Replacing any framework causes many new problems later. For example, JPA is developed for data access independence but at this time you are limited only the features of JPA. IDE is a major development tool, but at this time we need an “Integrated Development Frameworks” environment within IDE. (Similar with ERPs that brought together enterprise applications under the same umbrella)

Choice is a good thing. I can get up and running quickly using Hibernate; others prefer Cayenne or pure JDBC. I wouldn't want to mandate just one, but Mr. Levent is correct that frameworks must adopt the role of an integration hub, and Tapestry (with it's very dynamic, very late binding Inversion of Control container) really fits that bill!

I also chafe at the mention of the IDE: we've been down that path before (.Net, JSF) ... what we really need are tools that work with a minimal amount of support from the IDE.

2- Avoids heavy-componentization:

In web architecture, desktop-like componentization is heavy and inflexible. Components in desktop applications were very successful. They utilized reusability and used in IDEs. In web applications, component model doesn’t work at the same form. Efforts to convert HTML (+JavaScript) into component model will not be successful as desired. This is because HTML is dynamic (DHTML), works on client and declarative (Declarative Programming). With heavy-components, we loose declarative programming to some degree. We loose “Web Graphics Designer” ability to edit web pages because of moving from design-time to run-time and moving structure information from HTML to programs (With losing HTML Editors functions). Web editor’s favorite structure place is CSS files so what about CSS componentization? Another problem is architectural. Web GUI has 2 runtimes; server and client (Browser). At previous years, web frameworks supported only server-side-only functionality. Then today we see client-side-only approach. I think best solution is balanced mixture of both client-side (JS) and server-side code with component templates (not hard components but light partial HTML+JSP+Servlet codes). I’ll not detail further, there are already many discussions about “Component-based” versus “Action-based” frameworks on the web.

I come down strongly PRO componentization; that's been the focus of Tapestry since day one. Mr. Levent is correct that as more of a page's content is encapsulated inside components, the high level templates (page templates, in Tapestry terms) start to loose their ability to be view stand-alone, outside of the running application. However, I'd rather trade productivity and consistency (and testability) across my application for this one "feature". Here's a better question: how hard would it be to set up your application to run for the designer? Tapestry can allow a designer to run the application and see changes in real time.

3- No new tag markup or page template:

Some web framework requires learning a new markup with no added-value. Your form inputs turn out to be strange tags. Finally, developers don’t understand HTML, JavaScript, CSS because no time left for this learning. Who will fix GUI errors? Frameworks should bring minimum or no new tags (instead we may prefer attributes). HTML tags with simple JSP expressions are enough (KISS). Isolating developers from HTML and JavaScript is not possible.

Tapestry really excels here, as the Tapestry Markup Language templates are just XHTML with a namespace for the Tapestry parts; and those can be limited to just a t:id attribute with all other details in Java code. I don't do it that way ... it's more work for little gain, but a purist can appreciate this.

Even at the opposite extreme, a "heavily" instrumented Tapestry template is still pretty light, with no true Java code (though a few proeprty names and expressions will show up).

4- No XML usage:

Heavy XML usage for configurations makes programs hard to develop, hard to understand, hard to test. One example is “Page Flow” information in XML files. Another example is bean configuration. Yes, pulling this information makes it flexible but who needs it? How many times your page flow changed? How many times did we utilize flexible bean configuration? What about source code readability? I don’t like “Dependency” so “Dependency Injection”. I think dependency is not free that you have to manage its subtleties. Here is my anti-pattern “Dependency Rejection”. XML can be used in other useful places like AJAX messages or data import-export.

Here's where I agree; only Tapestry templates use XML. Tapestry 5 did away with all other XML (except for the ubiquitous web.xml, which is only touched once, when first creating a Tapestry project).

I feel that Mr. Levent is really missing the boat here; properly used Dependency Injection is incredibly important. Dependency Injection is what makes a clunky dinosaur of a language like Java useful, scalable in complexity, testable, and extensible (via late binding). To paraphrase: Dependency Injection is like violence; if it isn't working for you, you aren't using enough of it!

Dependency injection is critical to source code readability because it allows you to easily break your code into small, focused bits that each perform a well-defined function. The IoC container's job is to put all those tiny, testable bits back together into a running application. Tapestry IoC and Guice do this with aplomb.

5- Has its own web GUI page elements:

Rich web elements (say light components) are generally found only in JS or AJAX libraries. Web frameworks should provide rich elements like; Calendar, Dialog, Menu, Popup, Progress Bar, List, Grid, Tab (With sub-levels), Master-Detail Windows, Child Windows, Record Navigator etc. Developers can easily extend these elements. We are still turning around simple features like table sorting, filtering etc. We should step ahead. There is still no desktop-like web grid components to use (I see only in JS libraries) that I mentioned in my previous blog post.

Tapestry does well here (and this requirement seems to contradict item #2). In any case, Tapestry has decent support built in, with lots of great 3rd party support.

Fundamentally, Tapestry is page oriented: the Ajax effects can be well integrated, but not the degree of either a Google Web Toolkit solution, or something entirely hand-tooled (on top of Direct Web Remoting, perhaps).

6- Code generation:

Code Generation makes “Rapid Development” possible. Every part of software should be generated (Generative Programming); CRUD data access classes, business code, controller code, and view pages. Code generation takes development one step ahead of “Drag and Drop” WYSIWYG editors. If web framework facilitates code generation, developers could jump to customization details of application instead of building everything from scratch (MDA).

I disagree here: I don't like code generation unless it happens at runtime. If you look at Tapestry's "scaffolding components" (BeanEditForm and Grid particularly), you can see this ethic: the application is dynamically assembled at runtime. Likewise, all of Tapestry's meta-programming happens by class instrumentation at runtime, without a tedious build stage.

7- Has its own GUI JavaScript library:

Another bleeding integration point is JavaScript libraries. JavaScript libraries are not fully-integratable with web frameworks. They try to solve the problem in client side. What we need is close cooperation with client-side and server-side. Most of web frameworks unfortunately have no or little JavaScript in their presentation layer.

Again we return to integration; Tapestry has a set of libraries built on top of Prototype and Scriptaculous. Many applications also bring in jQuery. They all mix together nicely on the client side.

8- AJAX support (Asynchronous Communication):

AJAX eliminates bothering page-refreshes. Web frameworks should properly blend AJAX functionality into their code architecture. AJAX requires server-side coding. As we make client runtime powerful with AJAX, GUI state management code is duplicated. For example, if we update and fill a combo-box with AJAX call then server-side bean that is bound to this element is not aware of this state change. We have to change server-side state as well. AJAX functionality should be implemented without code duplication (Another interesting trend is AJAX MVC).

... and the nature of component encapsulation is to allow Ajax without the fuss and duplication alluded to here.

9- Portable among application or database servers:

Application and database portability is not easy. In Application Server side, class loader policies change, session management changes, deployment model changes etc. In DBMS side, join clauses change, paging, and sequence generating changes. Web frameworks should provide portable packages for different platforms. On the other hand, some web frameworks have their IDE and Application Server (believe me even DBMS). I think we must leave this job to the famous bright products (IDEs and Application Servers in the market).

This can be a sore point; the servlet API doesn't specify a few important behaviors for Tapestry (that mostly show up only in a cluster). I'm not sure what a "bright product" is though? Any clues?

Tapestry does work on popular servers (Jetty, Tomcat, WebLogic) because it's careful to follow the Servlet API rules, especially with respect to careful use of the HttpSession.

10- Input validation:

Data input validation is a very important feature. If validation doesn’t occur in application, database error occurs. Database errors are not user-friendly. Some validation errors may not be related to database. Programmers need automatic validation according to database object metadata. Custom validations should be added if needed.

I agree, and add further, that validation should occur on the client and then be re-executed on the server. Once you escape from the web tier, the errors get uglier.

11- Bug-free:

Because of bugs in frameworks, all average developers become framework expert spending valuable time to figure out the problem. “Focusing business problems” is lost. I read many open source framework hacks and workarounds in many blogs which is not the task of developer.

As if proprietary code is bug free? This one gets my blood pressure up ... I can't tell you how much time I've spent stepping though WebLogic code, guessing at what's gone wrong (where a bit of source code would have helped). The alternative to Open Source is to still become a framework export, but pay through the nose for the privilege, and deliberately let yourself become helpless, in thrall to your vendor.

12- Handles exceptions user-friendly:

If error or exception occurs, user-friendly messages should be returned. Application programmer has some responsibility for this but web frameworks may ease this task.

Tapestry excels here; I strongly maintain that Tapestry's exception reporting is the best of breed, with a detailed exception report and lots of contextual data ... and the ability to easily turn it all off or otherwise customize what happens when things go wrong.

13- Eliminates double-click, double-submission problems:

Double-click may cause double-submission. Double-submission may cause unexpected errors in application (2 threads tries to do same thing). Web frameworks can eliminate this problem even in client-side without going to server.

This is on the wish list; certainly a little JavaScript to disable the form or submit button goes a long way here! But a better solution intercepts the duplicate submission and that requires some coordination across the server cluster, which is why it isn't in Tapestry yet.

14- Authentication and authorization support:

User login (authentication) is still developed by programmers without knowledge of SQL-Injection attacks. Web application authorization is still missing. Who will be granted for CRUD on which application etc.?(User roles, permissions) I am sure that in every enterprise web application, application authentication and authorization is re-invented.

Is it the role of the application framework to define your security constraints? In a very constrained world, such as content management system, these roles and their application is well defined. I the real world of real applications, it's much harder to pin down. I've worked on many apps that had somewhat intricate permission schemes, and the ability for some users to "jump out" of those schemes.

That being said, Tapestry's modularity means that a standard security library can just be "dropped in". That's what we've been doing at Formos; we use a standard permissioning system, based on page and method level annotations.

15- Security controls for web attacks:

Web frameworks should prevent web security attacks like; Cross-Site Scripting (XSS), SQL Injection, URL Manipulation, HTTP Injection, Session Hijacking etc. Web client data is un-trusted and open to tampering so this is why we can’t quit totally server-side validation for the sake of client-side validation.

Tapestry does a great job on these issue; XSS is virtually impossible, as all output generated by Tapestry is "filtered" unless you specifically ask Tapestry not to. SQL Injection can't occur in a world where you are using Hibernate or another layer to generated prepared statements (this isn't PHP!). URL Manipulation is also somewhat of a non-starter because URLs are linked to components and components are configured on the server side to perform specific functions. It's not like Struts or Rails where you can hack a form submission to turn your admin flag on!

As I mentioned earlier, Tapestry re-performs input validation on the server side.

There is a concern in Tapestry in that Forms store serialized object data on the client side. This is both insecure and inefficient. A future release of Tapestry will address this by either encrypting or signing the data, or by storing the data server-side and just sending a "token" to the client.

16- Reporting integration and barcode support:

Reporting integration is important. We need reporting products/frameworks integration. Would you use your data access objects in your reports? Would your reporting engine use the same JVM runtime? Barcode is not a general requirement but in ERP applications it is very useful (AI/DC Automatic Identification/Data Capture). Barcode printing, barcode reading and matching may be provided by your web framework.(What about RFID?) Would your reporting product support your application barcode?

This is one of those entries in the original blog that simply makes me wonder; Mr. Levent clearly works on a specific category of applications, but I certainly have never written an application that needs to know about barcodes. Barcode reading? What does that even mean in terms of a web framework?

17- Messaging and workflow integration:

Web frameworks may support easy integration with messaging (JMS) and workflow products. Workflow is one of major element of BPM (Business Process Management). In some middleware stacks, this is included (i.e. JBoss Seam jBPM). Web application frameworks may support business events and workflow activities. These events can also be used to feed messaging backbone (ESB).

Even the example here is odd, and reinforces my earlier points: JBoss Seam doesn't have built-in workflow, its the Seam jBPM module that integrates into Seam. So as long as you are good at integration, we're in the success zone. And Tapestry is great at integration.

18- Application to application integration (i.e. Web Services):

In Java, there is external system (EIS, legacy) integration API, which is JCA, but inter-application communication within same JVM is not standardized. Let’s say we have 2 applications and one should use some call other application code. There is no standard for this. Basic solution is just adding other application’s path into its class-path and then using other application objects. We developed an Adapter API for standardization of this. In one-application environment, this is not a problem but if many applications are required to communicate, it gets more important. You can even convert your APIs into web services when necessary (integration with remote or non-Java systems). Web frameworks may provide tools for web services code generation, deployment and monitoring.

Mr. Levent has moved, about here, from some strong goals and guidelines for a web application framework to a kind of development environment wishlist.

19- Admin application for run-time process and user session monitoring:

This is very important in point of user and system management view. What are my users executing at the moment? Which applications take longer to finish? Which users are on-site? Which pages are they surfing? In each session, which objects are they created? What are the URLs that a user requested? Which SQL statement did a user execute?

This is an interesting concept and one that could perhaps be implemented using Tapestry's various meta-programming facilities. I've definitely been thinking long-term about a Dashboard facility.

20- System resource management:

If your application runs big queries that require a lot of system resource (CPU, RAM, DISK I/O), we are faced the reality that resources are limited. If applications don’t restrict user processes, then system will consume its all resources and will not respond to even small processes. For the sake of system availability some user may be rejected by system. Web framework may have such limitation API’s.

This concept is a tricky thing to bootstrap; if your machine is truly strapped, it may have trouble just getting to the point where it can determine how strapped for resources it really is! I know of no general purpose web frameworks that have this kind of feature.

21- Cluster support:

When server load is high and performance is a major concern, load-balancing is required. Application server clustering will not suffice, web frameworks must support cluster architecture. One simple example is framework’s id generators. They will collide in clustered Application Server environment.

I'm not sure what framework id's he's getting at here. Clustering a servlet application is generally quite sufficient, and clustering Tapestry is even easier, as it is very careful about what data is stored into the session. Tapestry is also good at keeping mutable objects stored in the session "fresh" when they are updated, but mostly it stores many small immutable objects where other frameworks store large mutable objects.

In terms of IDs; session ids can have, for example, DCE ids that can be cheaply generated anywhere with a guarantee of uniqueness (they just tend to be quite large). Database ids are generated, efficiently, by the shared database.

22- Multi-database, multi-company, multi-window, multi-session support:

Application user may need to work on multiple database instances. One user may have to work with multiple companies. User may want to use multiple GUI windows. Web framework should handle or prevent state corruption among windows. User may need to work on the system with many sessions.

To me, this indicates a single application deployed, and perhaps "skinned", multiple times ... or represents a single application that is capable of connecting to multiple databases at the same time.

Multiple windows can be something of a challenge; a single server-side session is shared across windows. Tapestry can encode state into URLs, which is handy but ultimately limiting. I think in the future Tapestry has the best chance of dealing with this cleanly because there's the gulf between persistent page fields and the session, which allows Tapestry to arbitrate ... literally, store different values in the session for different windows, but the same user. Not something implemented today, but quite possible.

23- Internationalization:

If there are global users, then i18n support is important. One key aspect here is Application Server and DBMS should also support your localization.

Tapestry has greate L10N support; applications can have localized message catalogs as can individual pages and components. Templates and assets (images, stylesheets, etc.) can be localized as well. Tapestry uses your browser's reported locale, but this can be overridden programatically. In Tapestry 5.0, you'll receive a cookie with your "true" locale. In 5.1, the "true" locale will appear in the URL (which is more search engine compatible).

24- SSL support:

If web application is wanted to be secure in insecure networks, SSL-support is important. SSL deployment in HTTP Server would not be enough. Even if SSL is not used, frameworks must encrypt sensitive data between client and server, like user passwords.

You may mark Tapestry pages as secure, using an annotation. Tapestry will automatically use HTTPS when building links to secure pages, and will reject any attempt to access a secure page using insecure HTTP.

25- Document attachment:

In every enterprise application, document attachment is important. Users may want images, Excel documents attached to their application records. Every programmer first search for an upload utility then tries to understand server document folders. Instead, built-in functionality saves valuable time.

Back to a wish list and not a real framework goal.

26- Mobile device support (i.e. Internet Explorer Mobile):

If we want to plan mobile access to our applications, how can we do this with web technologies? Many mobile devices have built-in web browsers and we may run our applications in these browsers. Web framework mobile support would be very beneficial at such cases. Otherwise, you should explore mobile web browser limitations by yourself.

I have long maintained that an application for a mobile device and an application for a desktop browser are not the same application. Creating a useful version of an application for a size and bandwidth limited client is more than just choosing new fonts and omitting a few options ... to do it succesfully is a completely different flow, and therefore, a different application (or at least, a seperate corner of the application).

You often hear about a magic XST transformation (Coccoon, anybody?) where a single service layer could be vended out in multiple formats. But I've never seen one in practice that worked, scaled and was maintainable, never mind acceptable to end users.

27- Portal features:

Partial web components should be supported to use in Portals or external sites. In portal terminology, its name is portlet. There are many synonyms; Widget, Mashup etc.

Tapestry 4.0 was a great platform for Portlet development, that will return in Tapestry 5.2.

28- Scheduling:

Application task may be batched and scheduled. After task completion, users may see results.

I have long thought of a layer for Tapestry to leverage Quartz for this purpose. Again, Tapestry's current goal of being a comprehensive user interface layer (rather than a total vertical application framework, which is what Mr. Levent is looking for) has made some of these non-goals for Tapestry.

29- Keyboard hot-keys:

Users, especially old TUI (Text UI) users want keyboard hot-keys. Buttons, command icons should be bound to hot-keys. Web frameworks elements can support this instead of developing in every application.

This is largely a function of HTML and JavaScript, things well encapsulated by Tapestry.

30- Alerts between users:

Users may want to send messages to each other or system admin may want to send messages to users like notifying a shutdown or an application restart. This feature will be very handy.

Again, a wishlist item that could easily be implemented for a specific application.

Summary

Mr. Levent has brought up a number of interesting concepts, and a number of real oddities, in his quest for the "improved web framework" (in fact, he's looking for a vertical application framework with some very specific niche capabilities).

I can't say that Tapestry fits his bill perfectly ... but I can say that Tapestry would be my first choice to anchor the stack that would meet his needs. The most important features of his "better web framework" are already present in Tapestry today.

8 comments:

Unknown said...

Howard,

For #19- Admin application for run-time process and user session monitoring and #30- Alerts between users, you may want to have a detailed look at MessAdmin, which does all of those 2 points except SQL reporting (it's in the TODO list), and is completely independent of the application server.
Like Tapestry, MessAdmin makes careful use of the Servlet API to bring light-weight and robust server monitoring, perfect for production!

I see Web framework (Tapestry) and monitoring (MessAdmin) as orthogonal domains. Since you seems to be thinking at adding a monitoring dashboard, I would be very happy to chat with you on the subject.

Unknown said...

My experience with Tapestry is limited to 4.0.2 and 4.1, so if this stuff is covered in 5 great.

14 & 24 - In Tapestry 4.x adding an annotation to a class was lost during runtime Class generation. Can Tapestry now support calls into user provided Annotations for rights checks?

25 - I disagree with you here. If you need a file upload, that upload needs to integrate into the frameworks form model easily. Same on the generation of binary data, whether downloading a data file from application code, or generated pdfs from a reporting library that should be supported. In Tap 4.x a project I worked on found that quite difficult, and had to resort to coding their own Service, (with mixed results)

13 & 15 - A "token" to store client side data could go a long way to solve your double-click issue as well. Struts has had "token" support (though much different than your #15 idea) that expires after first use. However, you should also look at the Spring WebFlow framework where every request ends with a redirect, so a page reload does not resubmit data.

#19 I'm not sure I agree that this is a framework responsibility. This seems like a cross-cutting concern that should be handled by a servlet filter. And now looking at MessAdmin, it looks like it fits what I said.

ojintoad said...

You loose your belt, you lose your car keys.

Sorry, I know there's a billion more important things being discussed, but the fact that you perpetuated this mistake the cited author made in your own post compels me to correct it.

Michael Buckley said...

11 bug free
I can't remember a Tapestry bug. Usability issues, but no code problems besides the borked portlet lib. And having access to the source helps me resolve any gaps.

12 Exceptions
This was what sold me way back on Tapestry 3. We did a little bake-off internally, and the first time we saw the line-specific error trace with context we were sold. Tapestry is the first (only?) project I've worked with that makes a first-class design goal of supporting developers when they make mistakes.

15 serialized data
I've been meaning to make a different default squeezer (to replace serializable) that instead generates a random token into a map on the session. The primitive squeezers (int, String, etc.) aren't such a worry from a security standpoint -- normal application security can manage those. But deserializing an untrusted bytestream seems dodgy.

25 attachment
Don't sell Tapestry short. The Upload component and the IUploadFile are easy as pie. But the download situation is 4.1 is pretty sad. "Write your own engine service" is rough advice for a new user.

27 portlet
We've beat 4.1.6 into submission (and submitted a patch). The biggest problem we have now is transient state being lost between rewind and render. Lots of components (contrib:Table most notably) are very unhappy losing transient state. Tap5 is probably much better now with the bias towards redirect-after-post.

Now that we've paid the price, we're pretty happy using Tapestry 4.1.6 portlets in Liferay.

Anonymous said...

Hi Howard,

Regarding item 2 - Avoids heavy-componentization:

IMHO, ASP (the old creature invented by Micro$oft) was a bad idea and Sun had a even worse idea when they invented JSP. All related 'technologies' only promoted a number of taglibs which are simply conceptually wrong!

IMHO, the idea you promoted when Tapestry was invented is correct: you have a *process* which consists of (a) prototyping; (b) approval (can even be offline with fake data); (c) instrumentation and (d) code the server side.

The idea of having attributes (jwcid) looks fine to me and keeps the prototype alive for next iterations, keeping the same process of prototyping, approval, etc.

I was user of Tapestry2 a long time ago and I've done only some small studies with it. Unfortunately I never found a company embracing Tapestry. Recently I've downloaded Tapestry5 and now my own venture will have a website powered by Tapestry. But, for my surprise, I found lots of "t:" tags everywhere and I couldn't understand what it all was about: This is not Tapestry! This is just another bloody taglib! :(
Even worse, the tutorial starts "from day one" promoting the use of this weird taglib which I dont want to learn or use. Seems like the beauty of your original design was lost :(

After looking around, I found that Tapestry4 was using "jwcid" as usual.
Then I found your post stating that Tapestry not only satisfies the statement of avoiding heavy componentization but also does it even better, etc. It says that purists (like me) will have the option to use t:id, etc. Well... I need to learn Tapestry again before understanding what you are talking about.

Anyway, new users are not being educated to avoid taglibs and they are not being educated to adopt the *process* you designed in the beginning.
What I'm saying is that the original beauty of Tapestry was lost or at least is not being advertised with big capitals anymore and this is really bad because the process is what makes Tapestry different.

Kind Regards,
Long life and success for Tapestry!

Richard Gomes
http://www.jquantlib.org/index.php/User:RichardGomes

Unknown said...

Richard,

For quick coding and prototyping, I favor the thing you mistook for a taglib (it's not, but looks similar).

However, jwcid is not dead, it is merely hiding as t:id.

You can use t:id in a template, and defing the component type and parameters in the Java code via the @Component annotation (which applies to a field).

Thus if you are comfortable with the full separation, you can have it. Again, for tutorials and prototypes, there's less keyboard typing involved in using the Tapestry namespace elements to define the component types & etc.

Toby said...

Concerning "15- Security controls for web attacks". Assuming content is read from a database and inserted into the templates using a raw insert without escaping (as it contains HTML markup, such as paragraph or bullet points) - then injected inline javascript would be successfully inserted into the page and executed? This was a XSS attack could occur...

Toby said...

p.s.: inline javascript could could be injected into templates in the webapps folder...if a hacker manages to upload files to the server through other exploits of other software running on the same server? What about supporting the CSP (content security policiy) and adding a white list of allowed domains that may execute javascript code?