Achieving a runtime CPAN with Dojo's XD loader

James Burke

AOL / Dojo Toolkit

http://dev.aol.com
http://tagneto.blogspot.com

Say what?

• CPAN (Comprehensive Perl Archive Network): Directory and storage of Perl modules (mirrored), CPAN.pm to get modules (dev/install-time loader).
• JSAN (JavaScript Archive Network): A CPAN for JS code. Perl used to fetch code. Dev/install-time loader.
• Dojo: JS library with a runtime loader. XHR, same domain.
• XDomain (XD) Dojo Loader: can load code from any domain. Like CPAN.pm, but runtime. CDNs for module storage.
• Talking about loading JS logic modules, not XD IO for data APIs.

The Dream

• Load code on demand at runtime, allow after page load.
• Automatically load dependendencies.
• Anywhere on the net.
• Storage in the sky, with a map.

How?

• Use script tags, XHR is not possible.
• Basic idea (incomplete, no dependency mapping):
  

  function loadFile(/*String*/ scriptUrl){
    var element = document.createElement("script");
    element.type = "text/javascript";
    element.src = scriptUrl;
    document.getElementsByTagName("head")[0].appendChild(element);
  }
  

• Need fast, reliable, global server storage.
• Need a directory of code.

Why?

• Performance:
  • xd gives you more network connections (2 per domain rule).
  • Asynchronous loading.
  • Code can trickle in, so less to load up front.
  • Caching: many web sites can reference the same resource.

Why?

• Money: Loading script from someone else's site saves you money.
• Easy dev: Try new code quickly. Avoid doing server cycles: Download, Install, Rinse, Repeat.
• Ideology: Supports progressive rendering, unobtrusive JS. Async, no script tags in body.

Stone Cold Reality

• Browsers load dynamic code differently (after page load).
• No universal standard to specify dependencies (yet - JS2 units?).
• Huge trust issue, in reliability and security.
• But let's give love a chance.

The Plan: XD Loader

• Use dynamically created script tags (for xdomain access).
• Need a way to specify dependencies inside code.
• Allow code loading after page load (to get progressive/unobtrusive/fast).

Let's make an XD loader!

function loadFile(/*String*/ scriptUrl){
  var element = document.createElement("script");
  element.type = "text/javascript";
  element.src = scriptUrl;
  document.getElementsByTagName("head")[0].appendChild(element);
}

• Sweet, works cross-domain.
• Supports progressive/unobtrusive coding styles.
• But...

Problems

• When is script loaded?
  • readystate, but IE cache issue. Use more tags?
  • polling, but hard to scale approach.
• Different eval order for head.appendChild(script):
• DOM order (Firefox/Opera)
• Network order (IE/Safari)
• How to resolve dependencies (foo.js needs bar.js, but you don't (want to) know that).
• Don't always have a HEAD

Dojo XD Loader

• Use Dojo build system: ./build.sh profile=0.9 loader=xdomain action=release
• Copies the file, saved as .xd.js
• Wraps code in a function.
• Extracts dependencies, passes them to xd loader function call.
• Before vs. After

Specify Dependencies

• Define a module resource (file): dojo.provide("foo.bar");
• List module dependency: dojo.require("foo.bar")
• djConfig.modulePaths or dojo.registerModulePath("foo", "http://some.domain.com/path/to/foo");
• Maps to http://some.domain.com/path/to/foo/bar(.xd).js

Path abstraction is nice.

• Uses URL for base path.
• Change URL in one place (djConfig/dojo.registerModulePath()) to change code provider.
• Allows switching from xd to non-xd very easily. Mix and match.
• Non XD:
  • sync XMLHttpRequest
  • Not cross-domain.
  • No build step needed.
  • Nice for dev.

XD quirks

• Must wrap code in dojo.addOnLoad(). Works after page load too.
• document.write bad.
• IFrame content needs local path djConfig option.
• dojo.i18n bundles in layered builds: use widget lifecycle function or dojo.addOnLoad()
• Subtle scope issue: use window.functionName not function functionName

XD loader details

• 0.9.0beta: dojo.xd.js is 26KB gzipped, dojo.js is 23KB gzipped
• Both contain functionality similar to basic jQuery or Prototype (plus module loader)
• The async nature and other benefits (more sockets, cheaper) balance it out.
• Runtime execution: does not use eval, does not seem to be a big cost.

Dojo XD in the Real World

• It Really Works (TM)
• AOL Webmail
• myAOL

The story so far...

• Have a workable xd loader.
• Have dojo.require/provide for code dependencies.
• Can produce xd code (.xd.js).
• Put it on a server, tell your friends.
• Can consume code: configure module path.

Scary second act

• Where to host the xd code?
• How to find code?
• Trust issues.

Finding a home.

• Have a fast delivery system w/gzipping and edge caching?
• If Dojo release: AOL CDN.
• If YUI release: Yahoo CDN. But no dependency specification.
• Amazon S3, costs money. jQuery (core) does this. Careful of latency and SLA.
• Dean Edwards used Google Code. Read the comments on the page.
• Work out a global system of high performing, free mirrors?

Matchmaker

• Where to find xd modules?
• Track your favorite site/blog.
• Use JSAN? Also allow users to download .xd.js code.
• Download might be nice for the next point...

The Elephant in the Room

• Trust. Continual (runtime) trust.
• Reliable servers?
• Is the code safe?

Reliable?

• Need high uptime for the code.
• Use reputation to regulate.
• Download the .xd.js code and host it on your own xd server.

Secure?

• Code provider can change code at any time.
• Reputation helps.
• curl + SHA + cron?
• Publish, code, publish digests?
• Download the .xd.js code and host it on your own.

Future of the future

• Load YUI, Ext, jQuery plugins with xd loader?
• Standardize dependency mapping.
• Global network hosting.
• Directory service. Perhaps JSAN.
• Standardize security model. Have browser do digest checking?

In the meantime...

• Try Dojo 0.9.0 on the AOL CDN (http://dev.aol.com/dojo)
• Easy to test drive.
• Look at TinyBuddy IM source to see Dojo 0.9 from the CDN, OpenAuth and Web AIM working together.