-
Get a monthly update on best practices for delivering successful software.
Eric Smith from 8th light gave a hands-on TDD presentation at last night's Chiphone meeting, hosted at Obtiva's downtown office, (conveniently located near the the train).
There was a good crowd of people, most attendees have 'played around' with iphone development, 4 have actively developed apps (3 people have live apps in the store). From my quick survey of those that have submitted apps, it seems most of them were free utility apps or simple games, with at least one commercial app Dash for Confluence. It also seemed that no one had yet needed to do any animation beyond the basics, with just a bit of core-animation, but no need for more lower-level openGL or animation engines.
Eric started off by saying that he's given talks on iPhone testing, but that just telling people what to do is not the same as letting them experience it for themselves, so we did a Randori, where a pair starts working on some code, and every 3 minutes one person from the pair swaps out and chooses his replacement from the crowd.
What I liked about this was that I felt like I got to know the audience better, and actually watch people reason their way through the code or a testing/mocking issue. (You know how sometimes you go to a user group, and it can be hard to get a chance to talk to others, or sometimes there is a 'know-it-all' guy, and you just want him to shut up. Knowing that you are going to have to go up there and code is a great way to silence those types)
When it was my turn, there was an interesting issue with one of the tests that had us all stumped for a bit, but ultimately ended up being one of those problems where you need to deconstruct everything and build it back up. (The issue was that while we were trying to set fooController.textView.text = @"foobar", we hadn't instantiated a textView object, or set it on the controller yet.)
Continue reading »
Some noteworthy Ajax Framework releases have come out in the last few weeks, along with some other news of interest:

Lots of exciting developments for Ajax developers and Web 2.0 entrepreneurs. I, for one, can't wait to see how the Google App Engine compares to EC2 for deploying and scaling Facebook applications.
Technorati Tags: ajax, dojo, google app engine, gwt, ext js, backbase, echo3, google search
Topics: Ajax Frameworks, Dojo, Echo2, Echo3, Ext JS, Facebook, Google, GWT, Javascript Libraries
Just as with advertising measurement for Ajax, I continue to follow the technology known as Comet (open up an HTTP request from browser to server, then keep it open, pushing content down periodically) to see if it is ready for prime time. I thought I'd share my reading list from the last few months:
While there is some progress, I have to say that Comet still seems to me like climbing up the down escalator. If you really want this kind of functionality in the browser, develop a new protocol and lobby for built in browser support.
Technorati Tags: ajax, comet, gwt, rails, dojo
Topics: Ajax Development, COMET, Dojo, GWT, Ruby on Rails
The new Google Gears comes with a long set of security warnings and disclaimers. Nitesh Dhanjani over at O'Reilly's ONLamp.com had some initial thoughts about the security of Google Gears:
If I had to pick ..., I’d guess that we are most likely to hear of existing XSS or browser vulnerabilities being abused to steal (or manipulate) Gears databases.
I agree with Nitesh: the attack vectors will remain the same, but the consequences will be much greater. Some folks, such as those working on the Dojo framework, are addressing some of the offline storage issues with encryption:
Right now Dojo offline adds a DES encryption library with functions to encrypt and decrypt. The example used encrypting/decrypting SSNs. He also pointed out that doing this in javascript can be computationally intensive and hasn’t been that feasible before. Right now though using Google Gears worker threads they can encrypt/decrypt in javascript at about 80k/second.
You could also roll your own from this implementation of AES, RC4, RSA and a number of other algorithms. While this approach solves the stolen laptop problem, it doesn't solve the XSS problem, i.e. as long as you can inject Javacript into a page, you can bypass this security. After all, the plain text has to pass through the sandbox, where it is fair game. In this case, Google Gears just papers over the shortcomings of browsers.
It is possible that WorkerPool could be used to limit the kinds of modifications that are made to the encrypted data:
The WorkerPool behaves like a collection of processes, rather than threads. Workers do not share any execution state. Changing a variable in one worker has no effect in any other worker. And created workers do not automatically inherit script code from their parents.
Members of a WorkerPool interact with each other only by sending message strings. Workers can also pass richer data types, by first converting objects to strings using JSON.
We can hide the encryption details in a WorkerPool member and communicate with it via messages. We still pass plain text data through the main browser execution context, but we can at least regulate the access to the encrypted database -- i.e. no bulk downloads or modification.
As a final thought, what's bad for ActiveX is also bad for Google Gears. The fact that the project is open source and thus not a black box, mitigates this criticism only somewhat. To make me comfortable with Google Gears, all of the XSS volnerabilities will have to be closed first.
Technorati Tags: ajax, google gears, security, dojo, encryption
Topics: Ajax Frameworks, BJAX, Dojo, Google, Security
Today we have the continuation of the interview with one of Dojo's creators, Alex Russell.
Ajax Challenges and Other Frameworks
Q: What do you see as the biggest challenge to Ajax and its further adoption?
A: Development of good, webish interface idioms and continued browser improvement and adoption. I'm excited that Microsoft is releasing IE 7 through windows update in a meaningful way, but unless we get a commitment from Microsoft to do the same thing for subsequent releases then I fear that dynamic in-browser apps will become a dead-end, held back primarily by the limitations of IE as a platform. From the scripter's perspective, IE 7 is a sub-point release -- clearly not nearly good enough.
Thankfully the Mozilla Foundation and Corporation are doing some great work in finally restarting some real competition.
Q: I have to ask the obligatory GWT questions. Is it a threat? A complement? Can it be used to integrate with Dojo or even write code for Dojo?
A: In order: Yes, yes, and yes.
Dojo is a set of client-side libraries that could be used with any server-side environment, including GWT. Since GWT is very squarely closed source and is tied to a single server-side language, we probably won't be the ones to write Dojo wrappers for it, but we hope that someone will (and we encourage them to!). It might come to pass that a huge library of stuff gets built up in the GWT community such that Dojo is entirely irrelevant. It's as likely as not. Interesting times ahead.
Q: Do you have any sense of how widely used your framework is?
A: We recently pass the 100,000 download mark for our last release, but we really have no way of knowing. Lots of server-side frameworks are starting to integrate it and therefore distribute it down to other developers who may or may not know or care that they're using Dojo. And we're all for that. Optimize for adoption, not control.
Q: What do you think of the server-side Ajax frameworks like Echo2 and ZK?
A: They're great, and we hope that more of them will integrate Dojo. We look forward to working with their developers to help them improve the developer and user experiences they deliver.
Q: Aside from Dojo, what is your favorite Ajax framework?
A: MochiKit. YUI is great code too, but MochiKit has it beat for clarity of vision and implementation quality. Bob Ippolito gets the constraints of the web. That, and I'm a Python guy at heart.
Security
Q: With various worms and viruses making headlines, the press is pointing the finger at Ajax as the culprit. You've said in the past that the fundamental causes of browser security problems haven't changed in the last few years. Still, there seems to be an uptick in Web 2.0 security exploits. Is that an accurate perception, and if so, why?
A: We're starting to see some new applications of old attack techniques, and in some cases like Jeremiah Grossman's new work, even some new techniques, but all of these things are predicated on the browser's assumption that all code running in the page sandbox is equally trust-able, and that hasn't changed and doesn't show signs of changing.
I'd welcome work on improving JavaScript as a language and browser implementations to support further application-controlled sandboxing and data hiding. The same-domain policy is far too coarse grained a solution for the applications we'd like to be able to write.
Comet
Q: You've done a lot of work on COMET lately. Tell me why you think polling isn't good enough.
A: Latency, load, and complexity. IM with a 5 second lag feels more like SMS than a real conversation. Also, systems that satisfy poll requests with a traditional process or thread model responder infrastructure have to throttle back the polling interval which again hurts the conversation. At that point, you wind up with folks trying to build notification queue infrastructures that are highly specific to their web server stack. At that point, wouldn't they have been better off with an HTTP stack that could just handle Comet?
Thankfully most things are gonna be there soon. The big challenges now will be at the application language layer. PHP and Ruby generally rely on process-level data isolation which translates into huge overhead for having lots of requests in flight. Twisted Python, Erlang, Java+continuations, and C# async network responders all can get around this pretty deftly, but making legacy code bases work with this stuff presents a challenge no matter what. We're trying to solve this in cometd by making the protocol stone stupid so that any language can have a client library that throws events at the event router (which may or may not be in-process or on the same box) for low-latency dispatch to the subscribed listeners.
Q: Wouldn't it be better to push for a protocol that was better suited to the task than HTTP?
A: I have tremendous respect for HTTP and for the challenges of getting anything new both into browsers and then out to users. HTTP isn't going away for this stuff if only because getting something else integrated into the fabric of the web is nearly impossible. HTTP has also shown itself to be highly resilient and extensible. The multipart/x-mixed-replace extension that Mozilla implements (and Safari almost gets right) allows the server to send discrete blocks over a single connection sans hacks. That's the kind of thing that makes me think that HTTP is a good enough tool for this.
Could we do better with raw sockets or some UDP network layer? Of course, but good luck getting that into browsers. There are options for doing this today, but they're all quite exotic and not for the faint of Ethereal-foo.
Q: Given that HTTP is a stateless protocol, many major web sites can get away with deploying firewalls and load balancers that can handle 64k or 128k open connections at a time. Won't COMET force these sites to retool?
A: Yes. But they were gonna need to do that anyway. Now that we can do C10K without too much effort, something was gonna force their hand. The great thing about Comet is that even if they do terminate these connections, you've still got the latency of polling. Worst case, it's neutral in terms of latency.
Q: What is Bayeux and where does the name come from? How does it relate to Cometd?
A: Bayeux is a name that Matt Trout came up with. There's a famous tapestry that chronicles, among other things, the Battle of Hastings, the life of William the Conqueror, and one of the first recorded spottings of Haley's Comet.
Bayeux is now the name of a JSON-based protocol for publish/subscribe event notification. It's not a descendant from the Open Source version of mod_pubsub, but in much the same way that we learned a lot of lessons in Dojo from netWindows, we're learning more about where responsibilities in the protocol should be split up thanks to the pioneering work of the KnowNow and CommerceNet folks (among others).
Cometd is a project to implement Bayeux in multiple server and client environments so that we can have easy-to-deploy Comet software for whatever your tastes may be. There are architectural constraints in many environments that will mean that a Bayeux server (possibly Cometd) will need to be run on a high port or on another server and we're trying to address those things in both the Bayeux protocol and in the reference implementations. We've got kind of a janky demo in SVN, but work on an initial draft of the protocol is moving forward at a good clip. I'm optimistic about having code that people can start to play with very soon.
Future Plans
Q: What are the biggest feature requests you are getting from users of Dojo today?
A: Better docs, widget system performance improvements, and whatever widget they love from [insert name of desktop GUI toolkit here]. Widget theming and templating also seem like common requests.
Q: What other web technologies do you think we should keep and eye on?
A: SVG and things that enable occasionally-connected operation. I'm also interested in seeing what Macromedia cooks up with Apollo.
Q: What is the Dojo Foundation? What is it's purpose and who is involved?
A: It's a not-for-profit 501c6 foundation that holds all the licenses for the Dojo Toolkit and OpenRecord projects, and soon the Cometd project. Like other Open Source foundations such as the ASF (Apache Software Foundation) or Eclipse Foundation, the Dojo Foundation provides development infrastructure, legal cover, and community-building tools for Foundation sponsored project.
If we had to do it all over again, I'm not sure we'd set up our own foundation again, but luckily folks like the ASF have dealt with many of the problems and gave us ready answers for some of these. There's an interesting thing with Open Source foundations, though, and that is that they seem to be language-centric. The ASF might be the one big exception in that they have both active C and Java communities, but I think that's the exception to the rule. We're still trying to figure out what we want the Foundation to be when it grows up, but I would be very happy if we can continue to provide a level, vendor-neutral playing field for high-quality web and JavaScript development for a very long time.
Right now the Board of the Foundation is just Dylan Schiemann and myself, but all the real power lies in the hands of foundation project committers. They can call votes on any topic at any time.
Q: How much work is it to support a major browser release like IE7?
A: These days, not a ton. We're lucky to have struggled in vain for eight or so years before working on Dojo so we pretty much knew what to avoid doing. Also, web standards have mostly done their job.
Q: What are your future plans for Dojo?
A: World domination...and trying to figure out ways to slip "a series of tubes" into interviews. Unfortunately we're doing better at the first.
All kidding aside, the road to 1.0 has a lot of hurdles still left to clear. Namely:
We've got active work in most of these areas today thanks in no small part to the support of SitePen, Jot, IBM, AOL, the Mozilla Foundation, and Google's Summer of Code program. That kind of broad community and industry support makes me very optimistic about our future.
Q: One final question -- has working on a high profile open source project been good for your career?
A: Yes. But working on low-profile open source projects was also good for my career. There's nothing like being able to show a potential employer a public SVN repo where they can see what you did or a mailing list where they can see how well you play with others. Potential employers would give appendages to get that kind of information before making every hire.
Topics: Dojo, Frameworks, Interview, Javascript, Javascript Libraries

A case can be made that Dojo is the single most influential framework in the Ajax universe. Any time I talk with Ajax developers about frameworks, the discussion always turns to Dojo. Even other framework developers take their hats off to the amazing work done by the guys at dojotoolkit.org, and many of them include Dojo as a part of their own frameworks.
I had a chance to talk with Alex Russell, one of Dojo's creators, about the framework's success and future. Alex had plenty of interesting things to say, so I've divided the interview into two parts. The second part will be posted tomorrow.
Dojo Background
Q: Alex, what's the history of Dojo? How did it come about? Who was involved in it's creation?
A: Dojo was the result of some fortuitous hiring by Informatica. They had already pulled me away from a job in security to help make their web-based BI (business intelligence) product more responsive and another team "needed an Alex", so I got in touch with most of the folks I knew from my days hanging out in comp.lang.javascript and on Dan Pupius' excellent DHTMLCentral.com. It turned out that, like me, most everyone had gone and gotten "real" jobs since no one wanted to pay a bloke to sit around and make things dance on a webpage in 2003. Getting back in contact with everyone inevitably started discussions. There was a sense of doom at that time around DHTML and JavaScript. Everyone had (re)invented the same things multiple times and I started to realize that at some point people reach a "personal effort radius". Even for people as persistent as DHTML hackers, there's a limit to how far one person can push something and in those day we only really shared code snippets and techniques. There weren't any "canonical" libraries despite many attempts (including my own) at it. The market didn't see a need.
The upshot of these discussions was a new mailing list which has subsequently become the Dojo contributors list. The original folks early on the list were Aaron Boodman, Dylan Schiemann, Tom Trenka, Simon Willison, Joyce Park, Mark Anderson, and Leonard Lin.
Fast forward to early 2004 and Dylan Schiemann and David Schontzler were also at Informatica and just as David was heading back to school we started on a new project that looked like it would need good SVG support. We needed a library that would be portable between environments, and Dojo was born. Retrofitting netWindows just wouldn't work. It was far too HTML specific. We tried to make a case for writing Dojo while working on this project and got some initial support, but in the end it was the folks at JotSpot and Renkoo that really helped push the project forward in the early days through their generous support.
By the time we had started on the new project at Informatica, we had gotten some momentum on the mailing list to try to combine all of the tools everyone had laying around. We set about to build a Standard Library for JavaScript. The original intent (and something we still hope to do today) was to get the best code and approaches from the community and boil them down into a single package.
We got all the IP donation stuff sorted out and it was off to the races.
Q: And the name "Dojo?"
A: The name "Dojo" came about because I had received a Cease and Desist from Microsoft when they noticed that my one-man DHTML project -- netWindows -- had the word "windows" in it. Of course, you can't appropriate a word out of the language, no matter how much money you throw at it, but Microsoft seems to think they can. When we went to name the project it was decided that we should pick something less likely to get us sued, so Leonard Lin proposed "Dojo," and compared to all the other dreck that was being suggested, it won easily. Naming things well is hard!
Obviously the recent past has lots to do with Ajax, Jot, IBM, Sun, the general dynamic-languages renaissance, but I've blabbered enough on this, and everyone always loves the creation myth more than the hard work parts anyway.
Q: What exactly was netWindows?
A: It was my first attempt at a DHTML library. It included things like a widget system, a theme system, a signals-and-slots event system, some useful widgets and a pre-XMLHTTP IO infrastructure using a request queue built on top of Iframes. There was even a primordial build and compression system. Some of the stuff in netWindows, like a generic focus manager, still haven't made their way into Dojo yet. Luckily we all got a chance to "reboot" with Dojo and take only the best parts out of netWindows, Burstlib, f(m), and some other codebases that have been donated along the way.
Dojo Strengths and Weaknesses
Q: What would you say are the strengths and weaknesses of Dojo?
A: We've got amazing infrastructure to support for folks who know and appreciate JavaScript as a language and know what they want out of a toolkit. Dojo is a set of layered libraries and a lot of folks love the ability to pick just the bits the like and ignore the rest, and while that's a strength for the folks who need that much power, it's a liability for everyone else.
In terms of weaknesses, the obvious answer is "documentation". We have a communication problem around trying to get the word out about how Dojo is layered and how to use it in real-world scenarios. We're working on those things, but some of it is inherent in trying to build something so broad within the constraints that the web imposes.
Q: Dojo has been criticized as being too heavy. How do you respond to that?
A: I'd respond "what are you trying to get done?"
In a lot of cases people assume that Dojo is some kind of monolithic thing. They download a release and go "OMG! It's huge! all these files, where do you start?" and I sympathize with that. In the past we've discussed breaking things up, but then you have the problem that a lot of other folks have today which is that you dump the problem of assembling the right versions of all these different things that work together and load in the right order onto the end developer.
The package and builds systems lets us avoid most of that, but we still ship the "src/" directory so that if you want some part of the library that's not part of a build, you can just say dojo.require("dojo.someModule.*"); and you can trust that it's going to work, that all of its dependencies will get loaded in the right order, and that you can make it faster later if you need to by dropping unnecesarry code.
We know of some people who use just the package system (around 20K, built) and people who do entire application UI's as Dojo widgets, and the system scales to both extremes. We ship the various profiles to give people people a head-start, but the system is flexible to the core. So the answer to the question is: Dojo is as heavy as your application requires and your interest in "right sizing" it lasts.
Q: What is your favorite feature of Dojo?
A: Hard to pick, but since I have to pick I'll say the build/package system. I hate Ant and I'm no lover of Java, but they've allowed us to solve a very hard deployment problem in a way that doesn't require anyone to change their code. Just make a build, point your app at the new "dojo.js", and your app loads that much faster. I love things that let you start developing fast up front and put the work of "making things fast" a deployment issue, where it belongs.
Q: There are client-side, server-side and various other classifications of Ajax frameworks. How would you characterize Dojo?
A: Pure JavaScript. We try to implement generic protocols for talking to servers such that we don't have a dependence on any particular server-side language or environment. As a result we've grown things like great JSON-RPC support. Of course, we don't have a strong identification with a particular group of developers the way Prototype/Scriptaculous or Atlas do, but that's fine by us. In our experience most good developers write in multiple languages on the server side, so why shouldn't their favorite JavaScript tools be portable? We've always had a mantra that I borrowed from Joe Kraus and modified a bit: "optimized for adoption, not control."
Q: What do you see as being the biggest competitor to Dojo?
A: Short term, the closed-source frameworks like Backbase and Bindows probably have the same breadth and depth as Dojo. Longer term, I think that we're in for an interesting time in figuring out if they're all anachronisms in the face $serverSideLanguage -> JavaScript "compilers". GWT doesn't make any sense to me since it's a less-powerful language being transformed into a more powerful one, but Python to JS and Ruby to JS look like real competition.
Q: Why do you say that Javascript is a "more powerful" language than Java?
A: Oh boy, I'm gonna get in trouble with this one.
Before I get into it, let me say that I write a lot of Java myself and don't view Java and JavaScript as being in opposition, and even less so with the introduction of Rhino into Java 6 (Mustang). Pretty soon most Java developers will get the language advantages of JavaScript with the library support of Java. The best of both worlds.
As a language, JavaScript is still horribly misunderstood. All real power in JavaScript comes from understanding closures, the "everything is always mutable" property, and the prototype chain. These are foreign concepts to Java developers who don't have FP or scripting backgrounds.
Even those that do, don't often recognize JavaScript as a language of that type. It's quite a good chameleon in that whatever you think it is, it's probably also that (or can be made to look like it). Making matters worse, there are a lot of idioms that the JavaScript community has cobbled togeather over the years to build classes that have some of the same properties as what's found in class-based OO, but fundamentally JavaScript is a different beast.
JavaScript is a functional language trapped in C-style syntax. It allows you to compose logic in ways that languages like Java generally require bytecode or precompiler antics to achieve. There has been some work in the Java world to introduce more powerful, more generic and more composable building blocks (e.g., jakarta commons-functor, but even that seems to have fallen into disrepair). In the end these efforts are always working against the grain of the language and even if they weren't, they face a similar comprehension gap among Java developers.
The JavaScript style of data hiding via closures is the flip-side of class-based OO: instead of having instantiable data structures that carry around behaviors, closures are behaviors that carry around their data with them. Thanks to the prototype chain and some magic in the "new" keyword, they can be made to look like class-based OO from a distance, but that illusion breaks down pretty quickly. Most Java programmers just assume that JavaScript is somehow "broken", when in fact it's just providing more abstract primitives for getting at the same end goals. Consider this highly contrived code:
function getThud(){
return this["thud"];
}
function ClassFoo(){
this.thud = "xyzzy!";
}
ClassFoo.prototype.getThud = getThud;
function ClassBar(thudValue){
this.thud = thudValue||"default thud";
// constructor assignment
this.getThud = getThud;
}
fooInstance = new ClassFoo();
alert(fooInstance.getThud()); // will be "xyzzy!"
barInstance = new ClassBar();
alert(barInstance.getThud()); // will be "default thud"
// prototype extension affects all existing objects
// of a class as well as new ones
ClassBar.prototype.getThud = function(){
return "prefix plus "+this.thud;
}
barInstance2 = new ClassBar();
alert(bar2Instance.getThud()); // will be "default thud" (!!)
barInstance3 = new ClassBar();
delete barInstance3["getThud"];
alert(barInstance3.getThud()); // will be "prefix plus default thud"
The implementation of logic can be promiscuous but object and closure scope can be used to make things immutable. The Ruby guys call this "open classes". I'm sure this code example will make strong-OO folks shudder in horror. OMGWTFBBQ! What if the implementation changes? How does anyone know that getThud is a member of ClassFoo? What about API versioning? Closures give us answers for all those things too but I'll leave that for another time.
The upshot of closures, the primality of the function object, and the prototype chain is that it's possible to better express your intent as a programmer in less space than Java and without resorting to syntax or contored class hierarchies. The language is just more supple. It bends more easily to your will.
There was a fascinating paper by Lutz Prechelt in 2000 entitled "An emperical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl". Among a representative sample of programmers, he found that scripting languages produced better performing programs that used less resources and required roughly half as much code as their Java equivalents.
Scripters lost out big-time in I/O (expected) and C and C++ developers could turn out code that beat all, but not with as much reliability as the scripters. The scripters on the other hand beat everyone for productivity despite roughly static LOC/hour productivity between the programmer populations. Takeaway?: if you're a decent programmer but want to look freaking brilliant, use a scripting language in a Java shop.
The paper wasn't a "do the same thigns the same way" kind of benchmark, it was a sampling of how programmers think about the problems they're presented with given the facilities of the various languages. In the 50's there was this paper called "The Magical Number Seven, Plus or Minus Two" that outlined the limits on human capacity for keeping things "on the stack" when solving problems. The limits are just as real today, and the mildly functional programming style that most scripting languages expose adds new tools for managing and encapsulating complexity to the programmer's toolbox. OO clearly isn't going away, but single-paradigm languages seem to be at a statistical disadvantage. I think we're seeing a realization that 20 years into the experiment, strict OO design is not the be-all and end-all of complexity management. Functional programming probably isn't either.
The hybrids (Python, Ruby, JavaScript, etc.) seem to have real legs these days. Luckily for the Java world, Sun is finally adding good scripting interfaces to the language. They're 5 or 6 years late to the party, but it's a big improvement for the lot of Java devs none the less.
JavaScript is poised to become a productivity shot-in-the arm for Java.
I can't wait to see how it plays out. Will Sun have the foresight and political will embrace the future? There were signs of it at JavaOne this year, but it was fleeting. From what I can see, the church still has the state cowed and functionally imobilized.
Tomorrow: Other frameworks, security and future plans for Dojo.
Topics: Dojo, Frameworks, Interview, Javascript, Javascript Libraries