Patch management between repositories, part 2

Back in 2008, I said the following:

I’m hoping a few people can point me to some nice freebie tools for applying patches in one repository to another repo’s code, and keeping the patches up to date. Or for handling cross-repository (and for that matter, cross-repository-platform) patches in general.

And…

From my brief research, Mercurial Queues seems perfect for this – within Mercurial repositories anyway.

Two years later, I’ve long since stopped caring about CVS. Both Verbosio (my experimental XML editor project) and Mozilla host their source code on Mercurial, and I’m getting better at Python. So I’ve once again solved my own problem.

The funny thing is that Mercurial Queues is both simple enough and documented enough for me to put this together. I mean, it was really only a day’s work to figure out. Once I found out .hg/patches was its own Mercurial repository, writing code to manipulate that inner repository was a piece of cake.

Whoever developed Mercurial Queues was very, very clever. Clever enough to make the basic design hackable. I like that.

Sparking some interest…

It’s been a whirlwind August, that’s for sure. Three weeks after Skyfire Labs, Inc. laid me off, I had a new offer from Mindspark – a sister company to ask.com. I started on Monday.

In addition, I moved from San Jose to San Leandro, so I could commute to Oakland more easily.

I still find it hard to believe, in this economy, that a guy with no college education could land a new job that quickly – and with a sizable raise, to boot!

I’ll be working on custom toolbars for Firefox at Mindspark.

Verbosio from 30,000 feet: Prelude

A few days ago, I realized, “This Verbosio project is so big, so detailed, that a high-level overview would be really useful.” These high-level ideas haven’t changed in the six or seven years I’ve held them, but there’s been no complete overview of the ideas at once. I decided, therefore, it’s time to take a step up – not back – and try to paint a comprehensive picture of what I’m really trying to do here.

(UPDATE: For those of you who don’t know, Verbosio is an attempt to build a prototype XML editor driven by add-ons – languages and user interfaces coming from extensions to the core architecture.)

I’ve tried before to explain it – and it’s helped in job interviews (more on that later) – but even a five minute summary just doesn’t capture the whole vision I have. Lengthy specification documents and blog posts don’t do it either – ultimately they end up out-of date and forgotten. So I’m going to attempt a new approach. I’m going to create an audio slideshow “fly-by” of Verbosio, explaining the concepts behind the project.

(I credit the JSConf X presentations Brendan Eich and Douglas Crockford gave as the inspiration for this. Slides plus spoken words in an online video is pretty effective.)

On Wednesday night, I wrote out an outline, in preparation for writing a script and slides. When I counted it out, seventy-five different slide items came up (give or take a couple). That’s too many for a single presentation, so as I write the script, I’ll break it up into a few smaller segments. Overall, this could easily run 90 minutes as a whole.

How serious am I about this? I bought a Logitech headset specifically for this audio slideshow, and I’ll be using OpenOffice and some freeware audio editing software to put it all together. That said, for those of you who’ve done audio slideshows before, I’d welcome any advice – including freeware recommendations – on the technical and conversational aspects. I’m dusting off techniques from my Navy Journalism classes – which were way back in 1996 – and my book, in 2001-2002. I could use a refresher course, so please: chime in on the comments!

Making a hash of things

Storing references to objects

As I’ve said before, I’m building a markup templates system. The idea is to
reduce markup into reusable patterns, and to separate output from the user
interface. The idea is that for every significant node in the output (and thus,
in the user’s markup), there’s a matching user-interface node, and vice
versa.

Keeping track of these node pairings is the challenge. In a simple case, you
have a DOM tree like this:

  <markup:template>
<markup:output>
<xul:groupbox>
<xul:caption/>
<markup:children/>
</xul:groupbox>
</markup:output>
<markup:ui-section>
<xul:groupbox>
<xul:caption>
<xul:textbox markup:xpath="xul:groupbox/xul:caption/@label"
markup:source="@value"
markup:required="@value"
/>
</xul:caption>
<markup:children markup:xpath="xul:groupbox/markup:children"/>
</xul:groupbox>
</markup:ui-section>

The markup in italics indicates one node pairing – between the output’s
caption element (actually, its label attribute) and the user-interface’s
textbox. Another pairing exists between the underlined nodes. The
<markup:output/> and <markup:ui-section/>
elements define yet another pairing.

So, ultimately, I need a way of referencing one node from the other.
(Specifically, I’d needed references to the output nodes from the
user-interface elements.) The most obvious way is to store each desired output
node as a property of the corresponding user-interface node – through the
setUserData() and getUserData() methods DOM 3 Core
provides. However, I scaled the template system upwards to include repeating
fragments (think for (var i = 0; i < x; i++) {...} in XML).
Since users can rearrange those fragments – or even create new ones and remove
existing ones – pretty soon, when sending the updates back to the output
section, it gets… complicated. Now, you have user-interface nodes in a
different order than output nodes, some output nodes without user interface
nodes (because those were removed), and some user interface nodes without
output nodes (because they were inserted).

More details in the extended entry.

Continue reading Making a hash of things

Symbolic links for everybody!

With a bit of free time on my hands, I started to apply a couple tricks I’d learned, and write a checkout and build Python script for my “unstable” Verbosio code – specifically the template editing system I’m still trying to get working.

One of the challenges is in making sure edits propagate both upstream into the Mercurial repository and downstream into the Firefox test builds. Since the code I’m working on starts two subdirectories deep in the repository, and ends up a different three subdirectories deep in the Mozilla source code, it gets tricky. Unless you use symbolic links.

Now, if you’re on Linux or Macintosh, you have a built-in symbolic link command already: ln -s (source) (target). Windows users… well, if you’re on Windows 7 or Vista, and using the newer filesystem (not NTFS), you also have a built-in symbolic link command: mklink /D (target) (source). If you’re on Windows before Vista, you can install an utility called “junction”, and off you go. UPDATE: Faulty statement struck per comments. Sorry about that.

On Windows and Linux, I find I only need one symbolic link: from my source repository’s desired working directory to a special folder in mozilla/extensions. On Mac, I need two: the source-to-target one, and another from target-to-inside-the-Namoroka.app-special-folder. (Otherwise, changes I make don’t show up in the Mac environment, short of rebuilding tier_app.)

Since I can’t always remember the format of these commands, a little Python takes care of the dirty work. (Oh, did you notice how Microsoft again does things in the reverse order Linux and Mac’s terminals do?) It also supports checking out the latest FIREFOX_3_6_*_RELEASE tag (handy when I don’t want to look that up either) and building the whole smash, including a few shell scripts I commonly use to run my Mochitests.

Wouldn’t it be nice if files that don’t get pre-processed were symbolically linked in at build time, until the installer was built?

I’ll blog again in a day or two, talking about hash tables in JavaScript – and why I suddenly decided to care.

Laid off from Skyfire

I’ve kept quiet about this for a couple days, in order for other layoffs @ Skyfire to not be surprised… but yeah, I was let go this past week.

So let me utter the only four-letter word I will about Skyfire: “Ouch.”

Skyfire was a wonderful company to work for – by far the best I’ve ever had. Now it’s time to update my resume and submit it to a few places, like the Mozilla Corporation and ActiveState.

On the plus side, it means for now I can go back to working on trunk code a little easier.

Giggle (or usability issue) of the day

I thought this was funny, and a true statement about the state of the Web today.

Fortunately someone at Mozilla is finally tackling problems like this, by implementing HTML 5 form controls and features. At a very shocking rate, I must say – it’s as if this has been his full-time job. I can’t keep up with him – I can only read the Bugzilla e-mails I get on it. Wow.

His name is Mounir Lamouri (volkmar on irc), and if you’re remotely interested in usability of web forms, you should keep an eye on him.

By the way, if you use a form on the web for anything: yeah, you should care. 🙂 He’s doing what I dreamed of doing three years ago, but never had the time to really do. (I was busy getting hired and learning C++ – so back then I was biting off more than I could chew.)

Help wanted: Visualizing a “broadcast / observer” model

Has this ever happened to you?

  • You have this great idea.
  • You understand it deeply – how it should work, how it should be used, etc.
  • You can write code to implement it.
  • You can write tests for it.
  • You can’t quite figure out how to put it in plain English, or in a diagram easy for your audience to understand.

Yeah, this happened to me. I’m trying to emulate XUL’s broadcast/observer model, but in limited parts of a document, and not necessarily in a XUL document. (The audience for this is fellow web and Mozilla developers.)

Basically, I’m adding in the concept of a “carrier” – as in carrier frequency, or relaying tower. (As an analogy, in a XUL document with inline frames and XUL subdocuments, each subdocument represents a carrier – XUL broadcasters in a subdocument can’t touch XUL observers in another subdocument. I’m attempting a similar boundary concept.) Broadcasters and observers live in the scope of a carrier node, but don’t cross the boundaries into other carrier nodes. Also, you can broadcast and observe not only attributes, but JavaScript values.

If you really understand the last two paragraphs completely in one reading, I’d be pretty amazed. So I’m trying to come up with an illustration of what I mean.

Here, the colored backgrounds indicate a carrier that the DOM node belongs to. (No background color means no carrier.) Some nodes within that carrier’s “grouping” have properties they broadcast… and other nodes in that carrier are observing on a particular name (“foo”, “bar” in this example), and picking up the properties. A carrier can observe and broadcast as a member of a parent carrier’s grouping, but not to any of the elements inside its own grouping.

Does this make sense, with the diagram attached? If so, great, and I’d like to know. If not, well, that’s why I’m posting this. I need some help, in making this easier to understand as a concept. That will help greatly both in using and testing the implementation of this model.

You can ask for the code implementing it and testing it, but be warned: it’s pretty hefty and intricate JavaScript. (Plus, I’ve already found a bug, and I’m still trying to figure out where it went wrong.)

(P.S. Yes, there really is a good reason for me to do this. Imagine a XUL grid where each row has a different number of columns. How would you allow the user to set the number of columns for each individual row? How do you make sure your XUL control for row 3 doesn’t affect rows 1, 2, and 4? This is just the most obvious example.)

Alex Vincent’s ramblings about Mozilla technology, authoring, and whatever he feels like.