Category Archives: Verbosio

Replacing JSLib in Verbosio

I came to an unpleasant decision last week. While working on reorganizing Verbosio’s source code, I realized that JSLib should not survive the transition.

I didn’t really like this very much. JSLib has been useful to me in the past. But when I thought about it, the only thing I used JSLib for was its file I/O utilities. These utilities should be used from chrome (and I abused that in my file search components).

Plus, I’m a much more experienced developer than when I first started with Abacus over four years ago. The Mozilla code base has similarly improved, and I just felt I could do the same job I was asking JSLib to do, with less (and cleaner) code.

I’m not saying JSLib is a bad library or a bad idea – far from it. I’m saying what I needed from JSLib is too small to justify having it around. Libraries for Mozilla are a popular idea – FUEL, STEEL, and JavaScript modules (.jsm) come to mind.

Besides, if you’re going to replace a library with one of your own, you should add value (not just cut costs).

I decided to write a library that, in theory, would be able to read and write to flat files, zip archives… and zip archives within zip archives. Think JAR files inside a XPI. Then, to make sure that I did it right, I decided to write a xpcshell test case for the module. (Yes, you can do that. The test harness doesn’t care whether you’re testing components or a .jsm file.) After a couple of days, I came up with FileCommon.jsm and test_FileCommon.js.

(The key is in FileCommon.getFile(), near the end. It takes multiple arguments, beginning with the path to a zip archive in the operating system, then an entry inside the zip (which is also a zip), then an entry inside the second zip archive, and so on.)

I’m sure there are bugs in this implementation. Right now no code in Verbosio uses it yet, and as I work on replacing JSLib with FileCommon.jsm, I’ll continue to add to the module. Eventually, I may make it a standard components module with new interfaces. Like so much else in Verbosio, it’s right now a proof-of-concept.

With JSLib, the time has come to move on. XUL Widgets will also fade, as I absorb its efforts into the Verbosio project directly. XUL Widgets (and Verbosio) doesn’t have any other customers, anyway, so centralizing under one roof makes sense.

Thanks for getting me started, Pete.

Regular expressions: Editing in a grid

This is for you, glazou, and your Diavolo editor. (Though I wrote it for Verbosio as well – I need it too.)

As I started my vacation, I was thinking, “I’d like to edit regular expressions in Mozilla. But I don’t want to edit one at a time. I want to edit a bunch, and see what they all do to a bunch of text.”

So in four late-night hours, I put together a simple XBL binding to help me do just that:

Each keystroke updates the table. When a regular expression is not well-formed, it disappears (and the results fields under it). When text and regular expression don’t match, the cell they share goes blank. The rest of the UI is, I hope, fairly self-explanatory for anyone familiar with JS regexps.

Yes, I know it’s not localized, and it’s not pretty. It’s functional, though, and for proof-of-concept code, that’s all I care about.

You may also notice this is on the new hg.mozdev.org repository. I’m in the process of converting over from CVS to Mercurial, and doing some other clean-up as well.

Patch management between repositories?

Gecko 1.9 is nearly here, and I’m looking forward to it. But there’s a catch… I have to make some fairly extensive changes to it, changes that will not live in the 1.9 repo on any branch. So I’m hoping a few people can point me to some nice freebie tools for applying patches in one repository to another repo’s code, and keeping the patches up to date. Or for handling cross-repository (and for that matter, cross-repository-platform) patches in general.

(The concept I have is that, following the new XULRunner build system, my app’s source would live in mozilla/verbosio – from a separate repository than the 1.9 code base.)

Continue reading Patch management between repositories?

Verbosio: Coming out of hibernation

Over the last several weeks, I’ve been having this gnawing urge to restart work on Verbosio. It’s been getting stronger, to the point where I just can’t keep quiet about it: I’m getting back into it, and looking forward to completing my work on an 0.1 “proof-of-concept” XML editor.

Since I put Verbosio to sleep several months ago, I’ve had a number of thoughts:

  • I want to use the new Songbird-provided build system. I’ve played around with it a few times since it first arrived, and I’m pleased. This makes it much easier to compile a specific XULRunner-based application, using Mozilla’s own build system.
  • I’ve gotten much more comfortable working with C++ code. When I started work on Verbosio, I had a goal that said “no compiling necessary.” That was because I didn’t want to muck around too much. The new build system – and a couple years experience – obsoletes that requirement, in my opinion. Besides, I still anticipate that I might have to customize Gecko a little bit. Hopefully not that much.
  • CVS would make me pay dearly for my earlier decisions. Because of the above two points, I want to rearrange Verbosio’s source code to take advantage of the build infrastructure Mozilla already provides. That means a lot of files and directories moving. In CVS, that means you lose all revision history. No thanks. Fortunately, SVN is coming, and supposedly soon. I am looking forward to it.
  • Writing to ZIP files changes the game. nsIZipWriter means that Verbosio, in its XUL demo work, could start working with jarred chrome for real. Fun times.
  • Bit rot has been surprisingly minimal. Code that I wrote many moons ago still works, at least in Gecko 1.9 beta 5. There was a regression in 1.9b4 that has since been fixed.
  • Waiting for OpenKomodo was a bit of a mistake. I figured by now they would be on the 1.9 code base, which Verbosio requires. Also, I haven’t reached “proof-of-concept”, that working model that lays the foundation for any possible merging or sharing of code. I need to move forward without them, for the time being.
  • I still have ideas to explore. In addition to this list, I’ve had a couple more ideas that might be useful. For instance, I’ve seen myself working with DOM TreeWalkers, where the NodeFilter is wholly implemented in JavaScript. Wouldn’t it be nicer, perf-wise, if you could have the first half of the filter implemented as a common C++ object (running ten times faster than JS code, thanks)?
  • I hope I can find the time and the motivation, again. It’s been hard, yes, but I’ve learned a bit about myself. These days, I can usually find the drive to work on side projects one, maybe two days a week. Yet these side projects are what keep my skills and ability to create, to innovate, at the sharpest levels. That’s when this business of software development is most fun: creating something new, useful, and radical. It also fires me up when it comes to my day job – I’ve noticed that my biggest ideas for work come to me either during or soon after a bit of heavy-duty cogitation on something Mozilla, but not Skyfire. I can’t explain it, but it works.

Ultimately, I’m still not sure of where this Verbosio project will take me – or where I’ll take it. But I take heart in the fact that I still don’t see anyone doing anything remotely like what I have in mind for Verbosio. Sure, it’s hard to do (as I’ve said before), but I have a vision, and that makes the years of effort worthwhile.

Thanks for reading!

Verbosio goes on hiatus

With OpenKomodo coming, I think I should really put Verbosio on hold for the moment. Komodo’s capabilities bring so much to the table that I’d be a fool not to look at moving much of Verbosio’s infrastructure over to Komodo instead of XULRunner. This means learning another platform (not as much fun as you might think), seeing what API’s I can benefit from, and what I can contribute back. Making sure OpenKomodo works on trunk XULRunner will probably be one of my main goals (since Verbosio uses several trunk features).

I am really quite pleased to hear this. It’s the kind of thing I’d want to throw a party for. I’ve been a big fan of ActiveState’s Komodo work ever since I was introduced to it at OSCON 2002. I kept thinking, “I want that!” Now, ActiveState is saying, “Here you go.” You just can’t beat that.

Verbosio’s on de facto hold anyway, as I’ve (once again) bitten off more than I can chew. If I could find a couple days uninterrupted, I’d be able to finish off the next piece of UI for it…

Nine days to Mozilla 24! Too bad we only get nine hours at the Stanford hall… anyone interested in hosting and/or joining me during the other fifteen for a hackathon?

Innocence (of scripting) lost

For the last few months, I’ve been doing Verbosio development based on Mozilla
milestones (as opposed to the trunk). Each of the last few milestones has
brought a change which impacted Verbosio. I didn’t spot any for 1.9a7, and I
thought the migration would be nearly painless.

There was a change impacting Verbosio – and depending on the point of view,
the result is either trivial or damaging.

Several months ago, I blogged about XPCOM class invariants for JavaScript
using the nsIScriptableInterfaceInfo interface. The code for this interface
lives in extensions/webservices… which was shut off with SOAP. Oops.

I filed bug 392004 to resurrect it, but even as I do this, I realize there’s
not much use for having it on by default. (Otherwise, people would have
complained about it much, much sooner.) Sure, it benefits me, but between one
developer’s convenience and millions of users’s Firefox installs, a few extra
kilobytes are expendable. Even for XULRunner 1.9, I expect it would be a tough
sell.

The simplest solution, of course, is to disable the invariant contracts when
they can’t be built. I should’ve done that in the first place. Invariants,
though, add valuable sanity checks. So to get them back,
I have to compile the scriptable interface info code as part of Verbosio.

There, I said it: “compile”. With that one word, I have overshadowed one of
those little goals I had for Verbosio from the beginning: that you could take
an unmodified XULRunner and run Verbosio. It’s ironic, considering Benjamin
Smedberg quietly taught Firefox how to run XULRunner apps like Verbosio – and
thus expanded the potential user base by a factor of five thousand. (I meant to blog about that in particular.)

Naturally, I can ship the necessary .dll, .so or .dylib files with Verbosio
– just pre-compile them. By all rights, I should. I’m not sure if I can ship a
DLL file compiled from Visual C++ Express, though – which means spending a few
bucks getting Visual Studio.

I have only myself to blame for all this – not just for relying on an
extension, but for not planning for C++ code to enter the picture. Sure, it sounds like a lot, considering this is for debug-only code. As a developer, though, I like having as much debug code on as I can get while I’m crafting the app. Live and
learn…

QAFound: Reducing testcases (follow-up)

I figured out the cause of my major woes tonight: bug 388267. I have a workaround, so I’m back to hacking on Verbosio’s trunk.

Also, our good friend Boris Zbarsky lent a hand debugging bug 388254, which is really very noisy in Verbosio. We believe that bug 388255 is a direct result of it, too. As for bug 388264, that is likely a duplicate bug, but no one’s checked that yet.

That accounts for all the bugs I filed as a result of my Verbosio snapshot a few days ago. I’m a happy coder.

Though also a bit lazy:

WeirdAl		is not at all excited about hacking layout...
		"you touch it, you own it" and all that, you know
		WeirdAl: the other side is "you don't touch it, it stays broken" :)
		hehe
ajschult	typically prefers not-broken

Verbosio: First Look

There’s an old saying in the open-source community: “Release early, release often.” I confess now that with Verbosio, my prototype for an extensible XML editor, I’ve done neither. I have put two years into it, and I have little to show for it – just some nice screenshots.

Until today.

A few days ago, I asked for help reducing testcases. Jesse Ruderman pointed out that I hadn’t given anybody a real testcase to reduce. Neil Rashbrook pointed out in response that Verbosio was too big for Bugzilla. In essence, both were right.

So I spent the last two days re-arranging the Verbosio repository, in order to provide a “snapshot” of Verbosio which people could reference in Bugzilla. It goes beyond that, however: it’s also a snapshot of the Verbosio code base which anyone with a XULRunner build (1.9a6 or trunk) can experiment with. It’s not just for filing bugs against Gecko, in other words.

It’s a chance to look inside my head, and see a piece of the vision I’ve been dreaming up for two years. It’s a chance to actually play with Verbosio, and see where it’s going.

Right now, it sucks, bad enough for me to say on the installation page:

This “snapshot” is specifically
for Mozilla developers in order to debug several problems which the author
believes are not the fault of his application (NS_ASSERTION failures, weird
XUL deck behaviors, etc.). If you are using Verbosio for any other
purpose, be aware that it can do nasty things to edited files, computer
performance, and your grandparents.
Use this snapshot with
extreme caution. You have been warned.

However, I suppose that as software projects go with initial source package releases, I’m in pretty good company. Netscape, I remember hearing, had a really ugly package for their Communicator 5.0 initial source release. Lots of other projects put out code that’s nowhere near ready for general use, with lots of incomplete features just waiting to shock users.

So, to get you started in Verbosio (if you’re daring enough), try the following steps.

  1. Get a XULRunner package (trunk or 1.9a6).
  2. Download the gzipped tarball (approx. 1 MB) from Verbosio’s installation page. (Note: it may not be on all the mirrors yet – it should be there in less than 24 hours.)
  3. Unzip and untar the tarball (after a virus and spyware scan – you don’t trust me that much, do you?)
  4. Run the application. Use the “default” application, not the “testing” application.
  5. When it’s running, go to Verbosio -> Open testing verbosio.xul. Watch Verbosio load a copy of itself (from the “testing” subdirectory) into its tabbed editing space. (Note: There’s a bug I haven’t figured out yet with Verbosio on the Mac platform, involving symlinks, which prevents you from doing this. Sorry.)
  6. Play with it to your heart’s content. As long as you don’t try to edit (opening up is okay) any other XULRunner applications or Firefox/Gecko-based XUL extensions with Verbosio, you should be pretty safe. The “testing” subdirectory application is disposable – you can get a fresh copy from the tarball at any time.

Several of the features:

These are just what’s visible to the user. There’s a lot of infrastructure too (ECMAScript design by contract, automated tests, support for extensions without conflicts of variable names, virtual files, a markup language for XML templates, DTD and entity management, undo/redo support including DOM operations, base URI mappings for starters).

Finally, the snapshot you see is on a branch (0.1a1pre.0.*), while the trunk is on a different line (0.1a1pre.1.* right now). That way, the branch itself can remain relatively static, and be a good reference for others. Also, the branch will work with any XULRunner build 1.9a6 or later (meaning 1.9a7, 1.9b1, 1.9, 1.9.1, etc.), while the trunk will remain based on a single milestone at a time (1.9a6 currently), so that I can work with an unchanging XR base. (In other words, the stable branch will tie to an evolving XULRunner, and the evolving trunk will tie to a stable XULRunner.)

This tarball and its contents are available under MPL 1.1 / GPL 2.0+ / LGPL 2.1+. Change log in the extended entry.

Continue reading Verbosio: First Look

QAWanted: Reducing testcases

I’ve gotten myself in a real pickle now. In writing this XULRunner app, Verbosio, I’ve ignored assertions, worked around oddities, but now I’m essentially blocked due to an obscure layout bug.

Essentially, I have a XUL deck, and what that deck is reporting as its selected panel is in fact not what the user sees.

Obviously, I should report it to Bugzilla. Except that I have no idea how to produce a testcase that will show the same results in Firefox 3.0 alpha 6 (the equivalent milestone). A bug that cannot be reproduced by a separate person is essentially INVALID, WONTFIX, and INCOMPLETE.

There are exceptions, but they require an exceptionally dedicated hacker (thanks, bz!!!).

So I am putting out a call for help. I need a few people who are willing to spend volunteer hours taking a look at these weird behaviors and trying to reduce them to manageable testcases which can be reproduced in Firefox 3 or XULRunner, either trunk or alpha 6. Once you’ve got such a testcase, please file a bug on it and cc me. If the bug is in Verbosio’s code, use Mozdev’s Bugzilla – otherwise, use mozilla.org’s Bugzilla. (You may find that both are doing something wrong in the same area, which wouldn’t surprise me.) I’m available weeknights and weekends, Pacific time zone.

If you are willing to do so, please leave a comment on this blog entry. I’ll gladly walk you through the process of reproducing these bugs in Verbosio (including building and running Verbosio). You’ll need a XULRunner build, 1.9a6 or trunk, probably a debug build – and that means compiling your own. You’ll also be getting your hands dirty with delta debugging in XUL.

I think the only reward you’ll get for reducing the testcases and filing the bugs is an “attaboy” from various people in the Mozilla community. That, and having your testcase added to Mozilla’s regression test harnesses, probably as a reftest. (For those near San Jose, California, it’ll probably also mean a free meal.)

UPDATE: I’m going to work tonight on putting out a “snapshot” nightly so that no one will have to run Verbosio’s make process. Then all you’ll need is a XULRunner debug build. News at eleven.

UPDATE 2: I’ve posted a gzipped tarball snapshot to Verbosio’s downloads directory. It’ll take a few hours for the tarball to propagate to the mirrors, but the instructions for grabbing it are here.

Ah, licensing…

So, the big news of the day for the open-source world (and largely cast to the side by Mozillians from my reading) is the release of the General Public License, version 3. One of the unanswered questions I have been thinking about is: “Should I upgrade Verbosio’s boilerplates to reflect a minimum GPL version of 3? Or should I leave it at 2 or later?” The consensus from the mozilla.org community so far has been this isn’t necessary. (Largely lost in the news was the release of the Lesser General Public License, version 3.)

This has some implications to think about, though.

The standard boilerplate for MPL/LGPL/GPL multi-licensed code says, and I quote:

Alternatively, the contents of this file may be used under the terms of
either the GNU General Public License Version 2 or later (the “GPL”), or
the GNU Lesser General Public License Version 2.1 or later (the “LGPL”)…

This essentially means there are now at least five different license versions which Verbosio (and by implication, most of Mozilla) code is available under: MPL 1.1, GPL 2.0, LGPL 2.1, GPL 3.0, LGPL 3.0. I’m not entirely sure what that leads to for my project.

There’s another point that I should mention here. One of my ideas for Verbosio is to include the GNU diffutils and Subversion software, integrated into Verbosio natively as XPCOM components. However, with licensing issues, I’m not sure I can do that.

The diffutils package is GPL only, version 2+. As timeless pointed out to me recently, the MPL license itself is not compatible with GPL (but does allow GPL as a parallel license). As for Subversion, well, it’s under a Collabnet license which – since I’m not a lawyer and I don’t want to think about it – I don’t know if it’s GPL-compatible. (Subversion’s Wikipedia article currently claims it’s under the Apache license in the sidebar.)

So the way I read it, I could release Verbosio with diffutils under GPL – but not under MPL or LGPL (with diffutils under LGPL). With Subversion thrown in, I just don’t know.

This could be a problem later on down the road, if someone wants to build a non-GPL application based on Verbosio. But that’s an old story, isn’t it?

Please, no holy wars over which license is better. I’d be happy if someone could analyze Subversion’s license and tell me if it’s compatible. Similarly I’d like to know if someone knew of a MPL- or LGPL-compatible diff / patch utilities package. I very much doubt everyone contributing to diffutils would willingly reclassify it under LGPL or MPL just for me.

UPDATE: I think I figured out a workaround for license incompatibilities. Namely, a license applies to what you ship in a single package. Extensions can come in their own packages, and thus with their own licenses. So for incompatible licenses, if I offer the new functionality as extensions (but not part of the main download), I can still comply with the licenses for everyone. (A key distinction is that the extension shouldn’t be something the main package or other packages with incompatible licenses depend on. Otherwise, it’s not really an extension or an external package, and the licenses are forced to work together where they can’t.)

I’ll have to be a little careful about how I arrange my repository, but this could be a workable solution. Please offer your professional opinions on this.