Recently a few of my coworkers visited the Add-On Conference in Mountain View, CA. (I elected to stay behind, and with our schedule it worked out nicely for everyone.) As most people reading Planet Mozilla are aware, it’s an annual event in December, talking about extensions for web browsers.

The Open Source Convention just put out another call for speakers for their July 2011 gathering in Portland, OR. OSCON actually hosts several smaller annual tech conferences as well at the same time. There’s a Python track, a PHP track, a Perl track (they must be planning to write Duke Nukem in Perl 6), etc. There’s even been, from years past, a small Mozilla presence there.

What if we made AOC a twice-annual event, once in Mountain View in December, and once at OSCON in July? I know it’s hard to prep for a conference, that it takes a lot of work to plan a speaking session. I’m just throwing the idea out to see who bites – particularly among those who spoke at AOC this year. On the plus side, if you’ve ever seen Damian Conway speak on Perl… well, it’s a treat.

At least then, a few of us could show up clean-shaven. :-)}}

Chalk this up as an idea that may have been done… or someone’s planning it and I don’t know.

Anyone familiar with the Einstein@HOME project? The idea’s pretty simple: you have a large number of calculations to perform, but supercomputers are a little expensive. Instead, you write an application which many home computers can download and run, and communicate with your main server.

I used to participate in it, but a couple features kept me off. First, I couldn’t share it between computers. Second, when I had to get rid of a machine, I couldn’t transfer the results to a new one. The final straw for me was when I found out a few years ago the Einstein@HOME project had accidentally been recycling data – sending me and several others data to crunch which had already been through the whole verification process. (What’s the point of leaving my machine on for days or weeks then?)

Now, imagine writing your distributed-computing application for a web browser. A little HTML, maybe some canvas work to make it look pretty (and show what you’re doing), and a whole lot of JavaScript.

Suddenly, you’ve got a lot of the necessary infrastructure built for you. Portable user accounts? Yep, we’ve been doing that for the last fifteen years (log-in pages). Communication with the home server? XMLHttpRequest. Fast calculation times? All the major browsers are implementing just-in-time compilation for frequently executed JavaScript – which is ideal for distributed computing (it executes a section of code very frequently).

There’s a few other benefits, too. It’s very hard to crash the platform in JavaScript – you have to go out of your way to do that. The whole WWW opens up to you, in a well-tested platform. Oh, and there are literally hundreds of millions of users of the major browsers. (Dare I say billions, plural? If not now, soon.) Distribution of your application becomes even easier than it is now: it’s just a web site. Plus, users can see when they’re on a secure connection to you (that little padlock in the corner).

When your application lives in a web page, your users can be fairly confident you’re not doing something evil to their machine. Security holes aside, outside the browser there’s few traces left behind. So it’s even environmentally friendly to the computer’s eco- and operating- system!

Of course, when you write something to force a JavaScript engine to do lots and lots of calculations, you’re going to start pushing the boundaries of what the platform is capable of. Which isn’t necessarily a bad thing: with Mozilla, you get an open, competent and somewhat responsive platform support community. If a bug is easy to reproduce – especially if it breaks the JavaScript engine (less so if it breaks something else) – and is filed in the right place on Bugzilla – and you’re nice to the community in question (politeness and patience helps!), you can get a pretty good response.

An obvious downside is that a distributed-computing web application could really wreak havoc as a botnet or a denial-of-service to the browser’s users. Here, the infrastructure of the browser could help again: for Firefox, Mozilla simply could put the dangerous application’s web site on their dangerous sites blocklist. Once that happens, and the user loads the web page, they see a big red warning page first, with a traffic cop icon scaring them off. The botnet disappears (or at least is seriously reduced in size, and thus, potency).

Commercializing a distributed-computing web application – where the end-user gets paid a little at a time for their results – might be practical too. (PayPal, anyone?) That could lead to a few dangers, though: who do you trust with tens of thousands of computers?

Right now, this is just an idea. I have no idea who might be working on that already, or if it’s just not practical. Comments welcome, though.

I’m sure everybody by now has seen Microsoft’s “To The Cloud” advertisement. I have just two things to say.

Do you really need a whole cloud computing set-up to edit a photo through cut and paste? And if you do, just what do you have on that computer?

If it was editing a video, I might understand the cloud benefit. Too much Batman homage, not enough respect for the audience’s intelligence.

While trying to resolve my “when should I retire a computer” dilemma, I realized I might not need to spend any money at all. After all, my overengined Windows desktop has plenty of capacity. Tonight, I found out how much.

Yesterday, I removed the Windows partition on one of my hard drives and replaced it with Fedora 13. After system updates, I did the following:

1. Downloaded ccache source code directly from ccache.samba.org. (Fedora’s ccache is version 2.4. This one is version 3.1.)
2. Unpacked ccache and installed it:

   tar -xzf ccache-3.1.tar.gz
   cd ccache-3.1
   sh configure
   make
   su
   make install
   exit

3. Added ac_add_options --with-ccache to my .mozconfig
4. Built a first pass of mozilla-central:

   cd mozilla
   make -f client.mk configure
   cd ../fx-opt
   time make -j4
   

   Total time: approximately 20 minutes. Finally, it runs fast enough for me not to worry.

5. Moved the old objdir to a new folder and repeated the previous steps exactly.

The total time on that is astonishing:

real    3m39.293s
user    3m49.130s
sys    1m53.586s

Given builds in less than four minutes, I should’ve done this a couple years ago. A couple other thoughts:

• I tried pymake -j4 as suggested in my previous blog post, but it failed to build.
• I actually tried to avoid putting Fedora on the same box as my Windows system, thanks to the inconvenience of a boot loader before. With the computer I have now, it turned out to be a non-issue. The system still defaults to booting Windows, and by hitting F10 at boot time, I can tell it to boot Fedora instead by selecting the second hard drive.
• I don’t think I need a solid state drive anymore…
• Nor do I think I need to try tinkering with a RAM drive.
• Anyone interested in a rarely-used Linux box built three years ago? Windows not included. I am willing to donate to a good cause…
• I think I’m gonna have fun with this thing.

I’m thinking about replacing my existing Fedora Linux desktop. I’m really on the fence about it.

On the one hand, it’s never, ever caused me any major problems. I don’t turn it on very often, except when I’m working with something I can’t easily do on the Windows box (I keep forgetting MozillaBuild doesn’t provide everything). Plus, I finally decided to try ccache, and it makes for an amazing build time: about 10 minutes on this three year old box.

On the other hand, it is three years old, and a build of Mozilla Firefox 4 trunk takes a couple hours when ccache’s cache isn’t up to date. (For comparison, the Windows box running Windows 7 now takes about the same time, and its specs are significantly better!) Fedora did a pretty bad job partitioning the 80GB drive, so at one point my home directory partition had only a couple GB of free space on it (while Fedora reserved about half the drive for itself and used very little). I bought an external hard drive to compensate – which turns out to be a pretty good idea anyway from just the backup standpoint. Most ironic of all is that the CD burner in the drive doesn’t reliably burn Fedora CD’s, so every time I want to upgrade Fedora, I have to let my Windows machine do the work…

Now, I hear stories of i7 machines that can do clobber builds of Mozilla in fifteen minutes, and I start to think about greener pastures. As much as I think, Oh, I don’t really need it, I keep thinking about it.

So what do you think? If you have a perfectly good machine that’s just old, running Linux, do you replace it, and why?

The last few months I’ve tried submitting patches to Mozilla implementing some feature as a XPCOM component, I’ve been told XPCOM’s the wrong way to go. I don’t understand why, though.

A lot of my work – especially Verbosio – has been based on the idea that JS components work. I can clearly define the interfaces in XPIDL, with some type checking done for me by XPConnect. Ditto for scriptable C++-based components, which I write only when I feel I have to.

I’d really like someone to write a DevMo or WikiMo article (or failing that, a blog post), explaining best practices, the costs of XPCOM, and alternatives. There’s several areas to cover:

• C++ to C++
• C++ code called from JS
• JS code called from C++
• JS to JS
• Privileged code called from content (not escalating content to privileges, just code that runs with privileges but exposed through some public interfaces to content)
• Anything obvious that I’ve missed.

Many of my own blog entries have been all about leveraging XPCOM. Having it explained in a tech note could profoundly affect the way I do things.

(I know I sound like a jerk in this article, and the tone is a bit hostile. I apologize for that, but as usual, I’m having trouble editing my own words… I can write new words, but which ones?)

P.S. Thanks for all the replies to my previous post… very informative, very useful opinions!

You’ve probably heard it before: “Commit early, commit often“. It applies to teams, and it applies to individuals.

I agree with that advice. I started my “unstable” branch on the Verbosio XML editor project a few months ago so I could commit freely without breaking the hg trunk of my project. I’m glad I did – I now commit (and push) my changes on this branch two or three times a week without much regard to whether it works or not. It’s very useful when working from multiple computers.

A few moments ago, I wished I had been even more aggressive about freestyle committing. I committed one change out of three that I need – another took me some four days to hash out, and the third I hadn’t started on yet. So I executed a “clean” command on my build script – the one that does symbolic links – and lost all the files in the source for that symbolic link. Four days of work, gone due to a bug in my build script and overconfidence in my set-up.

*sigh* At least I can take comfort that I’m not the only one to make that kind of dumb mistake.

Sometimes, even “Commit early, commit often” won’t save you. So commit very early, commit very often, and if you don’t think you can — create a private branch, and commit anyway!

My former boss at Skyfire once told me branches are cheap. Disk drive space is, too.

Back in 2008, I said the following:

I’m hoping a few people can point me to some nice freebie tools for applying patches in one repository to another repo’s code, and keeping the patches up to date. Or for handling cross-repository (and for that matter, cross-repository-platform) patches in general.

And…

From my brief research, Mercurial Queues seems perfect for this – within Mercurial repositories anyway.

Two years later, I’ve long since stopped caring about CVS. Both Verbosio (my experimental XML editor project) and Mozilla host their source code on Mercurial, and I’m getting better at Python. So I’ve once again solved my own problem.

The funny thing is that Mercurial Queues is both simple enough and documented enough for me to put this together. I mean, it was really only a day’s work to figure out. Once I found out .hg/patches was its own Mercurial repository, writing code to manipulate that inner repository was a piece of cake.

Whoever developed Mercurial Queues was very, very clever. Clever enough to make the basic design hackable. I like that.

It’s been a whirlwind August, that’s for sure. Three weeks after Skyfire Labs, Inc. laid me off, I had a new offer from Mindspark – a sister company to ask.com. I started on Monday.

In addition, I moved from San Jose to San Leandro, so I could commute to Oakland more easily.

I still find it hard to believe, in this economy, that a guy with no college education could land a new job that quickly – and with a sizable raise, to boot!

I’ll be working on custom toolbars for Firefox at Mindspark.

With a bit of free time on my hands, I started to apply a couple tricks I’d learned, and write a checkout and build Python script for my “unstable” Verbosio code – specifically the template editing system I’m still trying to get working.

One of the challenges is in making sure edits propagate both upstream into the Mercurial repository and downstream into the Firefox test builds. Since the code I’m working on starts two subdirectories deep in the repository, and ends up a different three subdirectories deep in the Mozilla source code, it gets tricky. Unless you use symbolic links.

Now, if you’re on Linux or Macintosh, you have a built-in symbolic link command already: ln -s (source) (target). Windows users… well, if you’re on Windows 7 or Vista, and using the newer filesystem (not NTFS), you also have a built-in symbolic link command: mklink /D (target) (source). If you’re on Windows before Vista, you can install an utility called “junction”, and off you go. UPDATE: Faulty statement struck per comments. Sorry about that.

On Windows and Linux, I find I only need one symbolic link: from my source repository’s desired working directory to a special folder in mozilla/extensions. On Mac, I need two: the source-to-target one, and another from target-to-inside-the-Namoroka.app-special-folder. (Otherwise, changes I make don’t show up in the Mac environment, short of rebuilding tier_app.)

Since I can’t always remember the format of these commands, a little Python takes care of the dirty work. (Oh, did you notice how Microsoft again does things in the reverse order Linux and Mac’s terminals do?) It also supports checking out the latest FIREFOX_3_6_*_RELEASE tag (handy when I don’t want to look that up either) and building the whole smash, including a few shell scripts I commonly use to run my Mochitests.

Wouldn’t it be nice if files that don’t get pre-processed were symbolically linked in at build time, until the installer was built?

I’ll blog again in a day or two, talking about hash tables in JavaScript – and why I suddenly decided to care.