Home ›

A New Hampshire Coder in Linden Lab's Court

Enable the Subscriptions block here!

Can Good Software Engineering Practices Work In Second Life?

Second Life

by James Turner
April 5, 2007

One of the most attractive features of Second Life, certainly one of the most powerful, is the degree to which the virtual universe can be tailored using the underlying Linden Scripting Language (LSL). It has certainly added scores of amateur programmers to the ranks, many of whom had never touched a line of code before the first time they tried to make a cube prim jump around the screen.

Because many of the Second Life citizens are first-time programmers, most of the existing documentation on LSL is written from a novice perspective. I'm sure that all the things that they need to get up to speed on must seem quite intimidating. However, from the view of a seasoned software developer, the experience is overwhelming in an entirely different way. There are so many restrictions and limitations of the Second Life scripting engine that any reasonably complex project is more about shoehorning into tight resources than anything else.

Forget dependency injection, aspect-oriented programming, efficient garbage collection, or arrays and hashes, for that matter. The training wheels are coming back on in a big way. But if you work at it, you can build some fairly amazing things in Second Life, and you don't need to abandon all your best practices to do it.

In this article, I am not going to make any attempt to provide a tutorial on LSL. For one thing, there are more than enough introductory tutorials out there, as well as the LSL Wiki. Instead, I'm going to focus on Second Life coding from the perspective of modern software engineering, and what techniques, if any, we can bring to bear to tame the Second Life beast.

JavaScript After a Lobotomy

At first glance, LSL looks a lot like JavaScript. But the differences become quickly apparent as soon as you start to do anything meaningful. There are no objects to speak of, just basic primitive data types. The only collection-like data structures are lists, which are strictly one-dimensional and cannot be modified in place. For example, the simplest way to remove an element from the middle of a list is to say:

oldlist  =  llListReplaceList(oldlist, [], 3, 3);

which can be read literally as "replace the sublist of oldlist starting at position 3 and ending at position 3 with an empty list." And you can't do in-place modification, so you need to store the resulting list back into the original variable.

What this all adds up to is a data management nightmare. You can use strides to approximate two-dimensional arrays, but you'll also quickly learn that list manipulation leaks memory, so once you start doing a lot of inserting, removing, and appending, you may quickly find your script gagging and dying from memory exhaustion.

For example, consider the following fragment:

thelist = [ "a", "b", "c"];
thelist = thelist + "d";

versus:

thelist = [ "a", "b", "c"];
thelist = (thelist = []) + thelist + "d";

If you run each version in a tight loop, the first version chews up a ton of memory, even though the four element lists are being immediately thrown away. In comparison, the second version is fairly memory neutral after an initial dip. But, paradoxically, if you try the same trick with strings, i.e.:

thestring = (thestring = "") + thestring + "foo";

it actually eats up more memory than doing the simple append. Because memory is at such a premium in SL scripts, you need to think about the memory impact of pretty much any operation that gets run a lot. For example, I built a card table in SL, and needed to store a deck of shuffled cards. It turned out that storing it as one long 104-character string (suit and value as two characters) rather than a 52-element list of two-string values saved a ton of memory, and didn't leak more every time I did a shuffle.

There's a tradeoff involved, though. Code and data live in the same memory space, so if you spend too much code on more efficient data handling, you may end up down more than you gained. I make it a habit to wire in instrumentation that can report free memory in my scripts, and monitor it closely to see how changes I make affect the memory profile. I also frequently drop little fragments of code into a testing script that calls the code many times in a tight loop, to see how it affects free memory. Part of the problem, of course, is that there's no way to see the underlying code generated from the LSL compilation, so there's no way except for brute experimentation to determine the relation efficiency of various approaches.

At the end of the day, one tried and true method of dealing with the unavoidable memory leaks is to have the scripts reset themselves, either periodically or when they cross a specific memory threshold. Since scripts lose state when reset, you need to use a trick to avoid having the object go back to step one (not a good thing when, for example, it's storing how much a player has in poker chips). You need to have a temporary persistence script running to one side, which the other script can send its state to just before resetting, and request it back right after. Typically, there are only a few hundred bytes of critical data that need to survive the reset, and the persistence script has so little in it that it will probably never need to be reset. By doing this, you can make sure your memory profile for a given script never goes critical, without losing track of what's going on.

Working Around the Limitations

As soon as you start doing anything complex, you'll run out of room in a single script, no matter how hard you try. Luckily, you can put more than one script inside the same object. But once you start trying to coordinate between two different scripts, you'll have to figure out how to get them talking.

For scripts physically in the same set of linked prims, you can use llMessageLinked. There are two nice things about llMessageLinked; first, that it doesn't incur a performance delay, which llSay does. Second, you can pass a wealth of information, because you get to pass two strings and an integer on each call, as opposed to a single string for llSay.

I've found that a good technique is to treat one of the strings as a list, and the integer as a message code. Here's an example from my card table:

sendMessage(integer destination, integer id, list message) { 
  llMessageLinked(LINK_SET, destination, llList2CSV(message), (string) id ); 
}

This is a helper function that allows me to send a list of arguments from one script to another inside an object. The corresponding event handler is:

link_message(integer sender_number, integer number, string message, key id) { 
   if (number == RECEIVE_MESSAGES_FOR) { 
        processMessage(llCSV2List(message), (integer) ((string) id)); 
   }     
}

The gimmick here is that I give each script a unique number (the RECEIVE_MESSAGES_FOR value), and the scripts specify that number (the destination argument) when they call sendMessage. Since the link_message handler is going to go off on all the scripts inside the object, you want to bail out as quickly as you can on the ones that don't care about this message, for performance reasons.

Each script also implements a processMessage function, which actually handles the script-specific functionality for the message. So processMessage acts something like an interface, required to be defined in any script that uses this message passing scheme. An example of a call using this technique is:

sendMessage(SEND_TO_SHUFFLER,  SET_CONFIGURATION, [ NUMBER_OF_DECKS, TABLE_NUMBER, DEBUG ] );

which says to tell the shuffler script to set its configuration to a certain number of decks, that it's inside a certain table number, and whether to use a static debugging deck for testing.

Scripting Across Objects

Once you start trying to communicate between two objects--say, between a heads up display (HUD) and the object it controls--you've doubled the number of ways things can go wrong. Either a bug in the HUD, or in the controlled object, can cause things to break. For this reason, I like to build "busy boxes" for testing. With a busy box, you can control exactly what you are sending to one of the parties, and verify that you get the expected result. It's also much easier to set up test scenarios, because you can hardwire specific sequences of events into the busy box, something that might be hard or impossible to do with the actual object under development. It also reduces the amount of testing code in your finished good, which helps with memory issues.

Luckily, busy boxes are easy to build in Second Life, because peer-to-peer communications is largely unsecured. (This is a good thing?) The only reliable and speedy way for objects to talk to each other is through llSay and listen(), using negative numbered channels. Since anyone who knows the magic channel can send a message to the listening script, you can easily write a test script in one of those ubiquitous pine cubes with a simple llDialog menu to run it.

A commonly used technique is to use llFrand to generate a random negative channel number and then use that to communicate. There's a big "yes-but" associated with this trick, however. It works great inside the same script, to pick a channel to use with llDialog, since the dialog talks back to the same script. I've seen people try to do it between a parent object and an object that's been newly rezzed, passing the channel number in via the on_rez event. This works great, until one or the other object gets reset, and loses state. Suddenly, the two scripts are incommunicado. For that reason, I always use a static negative channel number for communications between non-linked objects. If you're concerned about security or crosstalk, you can pass in magic tokens to filter out the junk.

Long-Term Persistence in Second Life

There isn't any.

Long-Term Persistence in Second Life, Take Two

No, really, there isn't. You can't modify the contents of a notecard from a script, and scripts lose all state when reset. Even if you trust a separate object to act as a persistence store, you can't depend on it not to be reset.

There are some extremely clumsy ways to fake persistence. You can rez objects, and use their very presence to indicate state. If you're going to store more than a few bits of data this way, you better have a huge prim allowance, though. You can use the positions of objects to indicate state. Beyond that, persistence means leaving the confines of Second Life, and using llHttpRequest requests to poke at an HTTP-based persistence layer.

I can't begin to enumerate the ways in which this is a bad thing. For one, you're introducing incredible latency into your code, which is hard to justify for anything but the most high-value, low-volume transactions. For another, if the website goes down, your objects are hosed, unless you have put a queuing mechanism in place. Since most of the data that gets stored this way is financial in nature, you don't want transactions dangling out in the middle of nowhere. At least for now, there's no good solution to the SL persistence problem.

Get a Good Tool in Your Toolbelt

From a software development perspective, Second Life is a nightmare. It's almost impossible to write unit tests (busy boxes are about as close as you get). Microtests are out of the question. There's no runtime debugging available, and the editor is pretty minimal.

There's only one really bright spot in the whole picture, and that's the fact that there's a good Eclipse plugin for editing Second Life scripts. Written by ByronStar, it provides basic error checking, code formatting, code completion, and function documentation for LSL. You can get it at byronstar-sl.sourceforge.net; it's currently under active development and he's been very responsive to bug reports.

Beyond the fact that it's a better editor than the native one, it's also a good practice to keep all your scripts safely outside Second Life. It's just too easy to fat-finger a delete of an object and accidentally blow away a toy full of scripts that you've spent weeks working on. Unfortunately, you have to copy and paste from the Eclipse window to the Second Life editor, but it's more than worth the pain.

Looking into the Future

The development buzz for Second Life is all around Mono. Linden Lab has announced that they are moving away from their proprietary scripting engine implementation, and towards having scripts compile down to Mono. The immediate benefit coders will see is in performance: the speed improvement is rumored to be an order of magnitude or more. Unfortunately, at least at first, that's as far as it will go. From what has been publicly discussed, there won't be direct access to C# or Java programming in the initial cutover.

However, assuming the memory restrictions are also lessened somewhat, once these languages become available for scripting, all bets are off. I could have written my card table in about a tenth of the time if hashes and arrays had been available for me to use. The rollout of Mono could spell a true renaissance for Second Life scripting. I can hardly wait!

References

» Submitted by Developer Network on April 5, 2007 - 11:02am.