Competing with pen & paper

In tonight’s linear algebra class, I made the mistake of leaving my paper notebook home. Ok, I thought, I’ll just use Amaya and see how that goes.

Not so well, it turns out.

Twenty minutes of lecture equals a frantic “where is that thing?”, and nothing learned…

  • The template for a MathML subscript is in a different panel from the template for a MathML summation (“sigma”), and you have to switch between panels constantly.
  • If you want two subscripts (and in linear algebra, two subscripts for an element is common), you get a modal dialog.  (Really? How many subscripts does an element need?)
  • Where’s the special “M” symbol for matrix spaces? (I’d post it, but WordPress eats everything after that U+1D544 character!)  We can get the real number set with ℝ..
  • The UI for Amaya is hard-coded, so I can’t change it at all.
  • Amaya’s copy & paste support is terrible.
  • It takes about 2 seconds to write [Ai]1j with pen & paper. In Amaya that takes somewhere around ten seconds, plus the dialog I mentioned earlier.
  • Oh, and the instructor’s going on, keeping a pace for students using pen & paper… there’s no chance of me keeping up with him.

After twenty minutes of trying to quickly jot down what he’s saying, without comprehension, I end up with some symbolic gobbledygook that’s probably about 80% of a match to what the instructor is actually saying.  But what I was able to write down was complete nonsense.

I ended up switching to scratch paper and pen, where I was not only able to keep up, but ask some insightful questions.

(Incidentally, I glanced at LibreOffice tonight as well.  I saw immediately that I’d have fundamentally the same problems:  unfamiliar UI and lots of context switching.  Too much to really listen to what the instructor’s saying.)

How does a computer compete with pen & paper?

Later tonight, I realized, if it only takes five quick, essentially subconscious penstrokes to draw Ai, and a fair bit of training to teach someone the equivalent keystrokes in an editor… then maybe a keyboard and mouse are the wrong tools to give a student.  Maybe something closer to pen & paper is best for quickly jotting down something, and then translating it to markup later… which sounds like character recognition.

Hmm, isn’t that something digital tablets and styluses are somewhat good at?  Maybe not perfect, but easier for a human to work with than a memorized set of keystrokes.

Now, I am starting to understand why computer manufacturers (and Firefox OS developers) are putting so much effort into supporting touchscreens:  because they’re useful for taking notes, at least.  Once again, I’ve somewhat missed the boat.

How does this impact my editor project?

The good news is this is way too complicated for me to even attempt in my proof-of-concept editor that I’m trying to build.  (The proof of concept is about making each XML language an add-on to the core editor.)

The bad news is if I ever want students to regularly use computers in a mathematics classroom (which is the raison d’être I even started working with computers as a child), I’m going to need to support tablet computers and styluses.  That’s a whole can of worms I’m not even remotely prepared to look at.  This raises the bar extremely high.  I’m writing this blog post mainly for myself as a future reference, but it means I’ve just discovered a Very Hard Problem is really a Much, Much Harder Problem than I ever imagined.

An insightful statement from a mathematics course

I’m taking a Linear Algebra course this fall.  Last night, my instructor said something quite interesting:

“We are building a model of Euclidean geometry in our vector space. Then we can prove our axioms of geometry (as theorems).”

This would sound like technobabble to me even a week ago, but what he’s really saying is this:

“If you can implement one system’s basic rules or axioms in another system, you can build a model of that first system in the second.”

Programmers and website builders build models of systems all the time, and unconsciously, we build on top of other systems. Think about that when you write JavaScript code: the people who implement JavaScript engines are building a model for millions of people to use that they’ll never meet. I suppose the same could be said of any modern programming language, compiler, transpiler or interpreter.

The beauty for those of us who work in the model is that we (theoretically) shouldn’t need to care what platform we run on. (In practice, there are differences, which is why we want platforms to implement standards, so we can concentrate on using the theoretical model we depend on.)

On the flip side, that also means that building and maintaining that fundamental system we build on top of has to be done very, very carefully.  If you’re building something for others to use (and chances are, when you’re writing software, you’re doing exactly that), you really have to think about how you want others to use your system, and how others might try to use your system in ways you don’t expect.

It’s really quite a profound duty that we take on when we craft software for others to use.

Compacting XUL interfaces?

For my day job, I work at a startup, basically as an expert in Mozilla technologies.  I love what I do, too.  But whenever I do user-interface work, I frequently run into a simple problem:  screen real estate.

Case in point, my latest work with XML entities and entity references on the Verbosio XML editor project.  (I can’t really talk about details of my work-related code here, but I can talk about the pet project.)  The demo panel for this, which doubles as a laboratory for my experiments, has four major sections.  The upper left corner holds a CodeMirror instance for one DTD document.  The upper right corner holds another CodeMirror instance, for a second DTD.  The middle horizontal section holds a third CodeMirror instance, for the XML document that might load the DTD documents.  Each of these has some ordinary XUL buttons on the right edge and a menulist, to help me control the individual sections.  In the bottom third, I have a XUL deck which can toggle between an iframe and a XUL tree showing the document object model of the XML I’ve parsed.  To the right of this XUL tree, I plan on adding a bit more, for showing the entities defined in the doctype or the entity references on a node.

EntityParsePreview

All of this UI lives inside a tabbox.  As you can see, it’s already a little crowded, though fortunately, it’s almost complete in terms of features I need.

I can’t do a fair comparison against the Mozilla Firefox user-interface; the main windows don’t have textboxes for source code or trees for DOM views. So their controls’ relative sizes don’t come close to mine:  they’re much flatter.

The built-in developer tools, though, do have an elegance to them, and are a fair comparison.  The right side panel, showing variables and events, can collapse away (and the animation’s pretty nice, too).  The left side panel has a listbox (I think) of scripts to choose from, and when you select one (either in call stack or in sources), the equivalent source code appears in the center.  Plus, they have some really tiny icon buttons in the UI, much smaller than the standard XUL buttons I use.  The devtools UI gives you basically what you need and otherwise tries to get out of your way.

Dear lazyweb of Mozilla UI specialists:  Can you point me to a developer.mozilla.org document with some guidelines for efficiently using the screen real estate?  My user-interface works, but I need some tips & tricks for making the most of it.  It could be simple stuff like shrinking buttons, or it could be creating new XBL bindings to masterfully present common ideas together.  I’m not willing to go to HTML5’s Canvas for this.  But my experience has largely been in components and JavaScript, not in crafting UI’s…

Or maybe it’s time for me to re-read Jenifer Tidwell’s excellent book from 2006, “Designing Interfaces”.   (I have the first edition.)

Think about what you do and to who you do it.

Think, for a moment, about the kind of man you throw out of office.   We’re talking about a revolutionary here.  A man who changed the way we live and work entirely.  A man who did everything he could to promote independence, writing missives that people listened to.

Yes, he had his faults.  What man doesn’t?  But he was a leader before he was the top dog, and he did very well as a leader.  He has laid his thumbprint on history with his works.  He put his heart and soul, and his reputation, on the line, day in and day out.

You might think I’m writing the above about Brendan Eich, and I am.  But consider this:  the same could be said for the third President of the United States of America.

Congratulations.  By the same rational thinking, we the people just threw Thomas Jefferson out of office because he once owned slaves.

FastEventLog.jsm: A quick and dirty event log viewing tool

About this time last year, I introduced a tree views module to my Verbosio XML editing project, which I’m still building infrastructure for.  One piece of that infrastructure uses the TreeViews module, and adapts it for simple sequences of objects in a common format.  Event logs were the use-case for this.

(This is currently independent of Mozilla Firefox’s own Log.jsm support, which is pretty nice itself!)

Let’s say you have a simple array of simple objects.  They all have a certain minimal set of properties.  If you want to visualize these objects laid out in a table, then each row can represent an object, and each column a property of the object.  XUL trees can build this table-like view, but they need some help.  First, they need a tree view that supports objects of the same basic type (TreeViews.jsm).  Second, they need to be built with the columns you want.

FastEventLog.jsm does this.  There’s two methods:  addPropertyColumn() and appendTree().  The first takes a property name, a label, the column width, and a couple other optional details.  The second method takes a box to hold the XUL tree element, a specialized “id prefix”,  a tree height, and the array of objects you want to show to the user.

If you have two separate arrays with the same structure, you can call appendTree twice. (Think expected results versus actual results.)

When working on my asynchronous transaction manager idea (more on that in a future post), I realized I couldn’t easily visualize what had happened.  The Firefox devtools debugger is awesome, especially with Thunderbird’s Remote Developer Tools Server extension.   But the debugger’s JavaScript object tree showed arrays of objects in an ordinary key: value hierarchy.  The data I wanted was buried very deep, and impractical to really analyze.  So I built FastEventLog.jsm and a helper XUL file to turn the tree branches I needed into XUL trees.

fasteventlog

The above is a screenshot with a bug deliberately introduced into the underlying transactions test to show what FastEventLog and TreeObjectModel (from TreeViews.jsm) can produce.  This little table view made diagnosing bugs in the async transaction manager very easy!

As usual, this FastEventLog.jsm is available under MPL / GPL / LGPL tri-license.

Promises: So We Rewire it!

We’ve been doing asynchronous code all wrong

The more I learn, the more I realize that old ways of doing things just aren’t capable enough.  For instance, JavaScript developers are taught to write asynchronous code using callback functions:

function asyncDriver(callback) {
    controller.goDoSomething(var1, var2, function whenDone(result) {
        controller.doSomethingElse(var3, function whenThatIsDone(result) {
            controller.etc(var4, callback);
        });
    });
}

This code is ugly, and hard to think about.  Wouldn’t it be nice to write:

function asyncDriver(callback) {
    var p = Task.spawn(function() {
        yield controller.goDoSomething(var1, var2);
        yield controller.doSomethingElse(var3);
        let result = controller.etc(var4);
        throw new Task.Result(result);
    });
    p.then(callback, reportError);
}

Why, yes, it would!  This is what Promises and Task.jsm bring to us.  It makes really messy code easy to read again.  The yield statements here force the function to pause in the middle of its operation, under the assumption that each value yielded is a Promise object.  When the promise “resolves”, the function continues with the promise’s resolved value.  A sequence of nested asynchronous functions becomes one function which looks synchronous (but is really still asynchronous).

This was what I used to write a JSONStore module for addons to use as a replacement for preferences – a way to store data and settings, and get them back when you need them.  There are some bugs, and this is just one of many ideas being floated for addon settings.  Discussion on the new module is ongoing in bug 952304 – it’s not part of Mozilla yet and won’t be for some time.  Use at your own risk.

But that’s not the whole point of this post.  I’m going a step further.

Promises, meet transactions

Promises are great when dealing with asynchronous operations… but if they fail and you want to roll back what has already happened, what do you do?  Being able to undo an operation is pretty important in a few environments, especially editing multiple files at once.

Now I like Mozilla’s native transaction manager API.  It works well for what it was designed for.  But it wasn’t designed for asynchronous operations.  Nor can I use the transaction manager in a chrome worker thread, because I can’t access XPCOM from chrome workers.  (There’s good reason for that, but it’s really unfortunate in this case.)

The transaction manager API has a couple other flaws:

  • Transaction objects don’t have any method for getting a human-readable description of what actually happened in the transaction.
  • If a transaction listener vetoes something, there’s no way for other transaction listeners which had already approved an operation to find out it was vetoed.
  • The transaction manager has limited ways of indicating its current state when things go wrong, not just in performing an operation, but in rolling the operation back.
  • If you’re dealing with multiple kinds of editing (DOM operations, source code changes), there’s really no good way to coordinate those.

As Tim Allen would say, “No power.  So I rewired it!”

I don’t have answers for these issues yet.  I think I’ll have to implement my own transaction manager API to improve upon all of the above.  But a design like that isn’t necessarily easy… I’d love to have help, especially if you think you might need something like this yourself.

One thing’s for sure:  when this new API and implementation is ready, it’ll have a lot of tests to go with it.  So, lend me your thoughts in the comments, please!

Enjoy the silence? Not so much.

For a long time, I’ve been wondering why no one besides spammers was responding to my blog.  After all, comments are one of the main features of a blog, and without it, a blog is just a bully pulpit.  Finally someone got through to me and let me know that comments were broken on my blog entirely.

*sound of a headdesk echoing across the Internet*

Wow.  Just, wow.  I had no idea.  I’m very sorry about that… and I can say that they’re fixed now.  (Thanks a lot, old silently busted WordPress theme!)

According to my admin panel, I haven’t had any comments on this blog in nearly two years.  So, if you’re curious about XUL editing or my Verbosio XML editor project, or about anything I usually ramble about, please take a few minutes to read over my past couple years of posts and drop a line.

Should I switch to GitHub? Should I mirror?

I’m in a dilemma.  Enough Mozilla community members have asked me, “why don’t you switch to GitHub?  There’s a larger JS community over there.”  I can’t exactly ignore that, considering that I really do need help.  And yet… I like SourceForge.  But not enough to be alone in the desert.

What do you think, Mozillians?  I know you’re busy, but how strongly do you prefer GitHub or SourceForge over the other?

UPDATE:  I just discovered comments were broken… fixed.

Introducing MicroXUL: A rapid XUL prototyping mini-project

(tl;dr) I’ve started a XUL editing miniproject, and I’m looking for volunteers to help build it out.

screenshot of MicroXUL editor's layout page

Which came first, the chicken or the egg?  It’s an old question:  conceptually, one had to exist to create the other.

My prototype XML editor project, Verbosio, is XUL-based.  XUL is an XML language.  So to edit XUL, I need a XML editor.  To build my XML editor, I need to edit XUL.  So which comes first?

Chicken, meet egg.  I’ve started work on a “MicroXUL” editing tool to help rapidly experiment with XUL user interfaces and XBL bindings.  It’s extremely primitive, but this is a proof-of-concept.

I’ve been using (and continue to use) the excellent Komodo Edit environment to develop pieces of Verbosio, including MicroXUL.  The screenshot above represents one wizard page where the user can type XUL markup into one panel, and see the rendered XUL in another, on-the-fly.  From this one wizard page and Komodo Edit, I hope to build out the rest of MicroXUL (and Verbosio) with a number of useful features:

  • XUL Project Details
  • Basic XUL Layout
  • Widget-specific wizard pages (based on analysis)
  • Images & Styling
  • Event Listener Declarations
  • Label & Access Key Internationalization
  • Keys and Commands
  • JavaScript controller object properties
  • Complete event handlers
  • Packaging of the XUL
  • Testing support
  • Documentation
  • Checklist
  • XUL menus shown through a XUL tree
  • XUL-to-XBL conversion tools
  • Editing existing XUL fragments from Firefox, Thunderbird, etc.
  • Editing MicroXUL itself
  • Drag & Drop of known XUL markup elements into the editing space

This project uses XULRunner (27+ required, currently in Aurora), Python 2.7 for building & launching, Jasmine 1.3 for automated testing, and CodeMirror 3.19 for syntax highlighting.

I started this because I realized I needed new XUL controls and XBL bindings to support new capabilities, such as a XUL user interface for the new Log.jsm architecture in Firefox 27.  I couldn’t use Verbosio yet, nor its templates system, since those haven’t been built out yet.  But I had just enough pieces from previous work, including my virtual:// protocol handler concepts from seven years ago, to put together a quick & dirty XUL editing interface.

Checkout and build instructions:

  1. hg clone http://hg.code.sf.net/p/verbosio/microxul/code microxul
  2. cd microxul
  3. ./project.py –update-sdk=aurora run
  4. (Mac users for the moment) ./project.py –browser (path/to/Firefox/27/Contents/MacOS/firefox)

Obviously, there’s a lot of work to be done on MicroXUL before it’s usable.  But the project is open under MPL / LGPL / GPL.  If you have a pain point with authoring XUL, this is a good playground to try fixing those pain points!  Your help and experimentation are both welcome.

Project page: https://sourceforge.net/p/verbosio/microxul .  If you need help and the SourceForge project pages don’t provide what you need, please leave a comment here and I’ll try to fix it.

Alex Vincent's ramblings about Mozilla technology, authoring, and whatever he feels like.