Dan Newcome on technology

I'm bringing cyber back

Archive for June 2012

Advanced remedial Javascript

leave a comment »

I’ve been using bits of code written by Douglas Crockford for years now. As you may know, as author of “Javascript: The Good Parts” he is quite opinionated about the language.

Crockford refers to some of these concepts as “remedial” Javascript. On one hand, yes, we are dealing with the most basic concepts of the language, but they can be a little tricky and easy to forget or misunderstand so I like to call them “advanced remedial” concepts.

I tend to have strong opinions about these things as well. There is one place where I have quite a conflicted opinion, and that is with the typeof operator.

I really like to use the native facilities of the language where possible, not just in Javascript, but in any language. It’s great to have a library of higher-level abstractions, but base language features like inheritance and type checking are things that I like to leave alone if possible.

Fortunately, simple prototype chaining is enough to handle inheritance for the bulk of my work, but deficiencies inherent in the typeof operator are really difficult to work around.

I’ve been using a one of Crockford’s functions, typeOf, published as part of his Javascript recommendations. As useful as it is, I have a need to identity another type that isn’t covered by default here: the RegExp (regular expression) object.

The problem with RegExp is similar to the problem with Array checking – typeof identifies them both as “object”. I added support for this using the instanceof operator, which is able to identify a regular expression. Crockford prefers property checks for arrays, so I might be missing something here. Alternatively, we could check for the exec() function and assume that we aren’t checking for something that also happens to have a function called exec.

I’m sure this isn’t the last word on finding regular expressions, so I’ll publish this as a work in progress, although I’m using this in several of my own projects now.

function typeOf( value ) {
    var s = typeof value;
    if( s === 'object' ) {
        if( value ) {
            if (typeof value.length === 'number' &&
                    !(value.propertyIsEnumerable('length')) &&
                    typeof value.splice === 'function') {
                s = 'array';
            else if ( value instanceof RegExp ) {
                s = 'regexp';
        } else {
            s = 'null';
    return s;

It is possible to “fool” (if you want to call it that, in many cases it would probably be correct behavior anyway) this method by subclassing RegExp, but it seems unlikely that this would be an issue in practice.

I didn’t see any tests published with the original typeOf, so to make sure I didn’t break anything I wrote some tests that cover the nominal cases and negative cases.

* Run test with qUnit
test("test-typeof", function() {
	tst( [], 'array' );
	tst( {}, 'object' );
	tst( "", 'string' );
	tst( 1, 'number' );
	tst( function(){}, 'function' );
	tst( /x/, 'regexp' );

	function tst( obj, type ) {
		var types = [ 'string', 'array', 'object', 'number', 'function', 'regexp' ];
		types.forEach( function( item ) {
			if( item == type ) {
				equal( type, typeOf( obj ) );
			else {
				notEqual( item, typeOf( obj ) );
		} );

I didn’t publish this separately on GitHub since it was so small so I’m publishing it here on my blog. I might try to publish a set of JS utilities including this one, but my feelings toward those types of personal util projects is mixed. Many times I see some open source project that makes extensive use of something like mikes-utils or something similar. As projects mature these are nearly always factored out anyway.

I’d like to embrace extreme composability, maybe mixing 10s or 100s of single methods together to make a dynamically configured library, but I haven’t seen any tools for making that happen reliably.

Written by newcome

June 25, 2012 at 11:25 am

Posted in Uncategorized

Meteor, magic and n-tier applications

with 2 comments

I’ve been shouting about how cool Meteor is recently, but as I’ve been using it I’m wondering about a few things. 

 As was pointed out by someone else in a blog post (which I can’t find at the moment), Meteor is essentially allowing us to go back to writing 2-tier applications like we did when client-server applications were de rigueur. For example, a line-of business application that ran on a workstation but connected directly to an Oracle database without any service layer in between.

The immediacy of this programming method was nice in the beginning, but once we had to deal with schemas and business logic, and mixing various versions of application APIs we learned that this is a pretty hairy maintenance issue brewing. The data lasts a long time in an organization, and new applications are generally built on the data/schema over time.

Contrary to the current conventional wisdom database administration is a real issue and not a trivial thing once you have a lot of complex data. Things like NoSQL really do nothing to bail you out here, the reasons for which are fodder for another entire post. Assuming that you believe me that data maintenance and migration is a tough problem and that throwing NoSQL at it probably isn’t going to fix what you want it to fix, what do we do here?

In the past, before we started sticking Web services in all over the place, the next logical step was to create stored procedures that ran in the RDBMS itself. Now we can expose an API over the data, which is consistent with its schema and allows us to abstract somewhat the actual table structure. Some of these things would be views on steroids, and in more complex scenarios we can do mapping to/from legacy schemas.

If you know me, you know that I’ve always had mixed feelings about running code in the database. Databases are complex enough when you consider indexing and various constraints that are defined on the data. Once you have code running there that does things like create temp tables or iterate through cursors (fetching data) you have potential performance problems, that are far more delicate than looking at a single query in a query analyzer. The database is also notoriously hard to distribute (this is actually one place where NoSQL can be a big win, but only if your app is conducive to it.

Ok so that brings us to what I really want to talk about – Meteor.js and things like Socket.IO and Now.js. All of these frameworks are JS frameworks designed to make server push and pubsub style apps easy to do. The difference with Meteor though is that it is designed to abstract the service layer away completely. The database looks like it is local to the client code, and so we are essentially writing what looks like a standalone app with an embedded database from the client’s perspective. 

 This has some huge advantages in terms of rapid development and deployment but by obscuring the layers of the application it makes reusing the database and server-side code very difficult.

When I first looked at Meteor, I thought of it like a realtime Rails. If Rails is any indication, eventually Meteor will be flexible enough to handle pluggable backends via npm modules, support APIs, etc. Only time will tell. For now I’m having a hard time deciding whether I’m going to double down on using it or not.

Written by newcome

June 11, 2012 at 2:56 pm

Posted in Uncategorized

Thoughts on UI frameworks

leave a comment »

I was playing around with Node-qt recently to see how difficult it would be to do a native-UI app using Javascript. I’m thinking of something along the lines of the Leo outliner which was done with Python and QT. I’m kind of interested in how using Javascript as a scripting language would look in something like Leo.

Once I started digging around in QT I realized that there were some controls that I’d need to extend, like the QTreeWidget, which weren’t exposed in the Node-QT API. I’d have to first extend the API for this widget and then dig in and write some C++ to create my custom version of it.

This got me thinking that while it is nice to have dynamic language bindings for a well-defined native library, I don’t think the paradigm works that well for all domains. If you need to parse XML, pretty much any task you can think of can be performed using the fixed API of something like libxml2. You probably won’t find yourself wanting to extend the library to do anything specific, you’d do that in the client code. Same thing with libpcre or something like that.

UI frameworks seem to be a whole different animal. You are lucky if you can get away with just using some standard buttons and input boxes, but for any nontrivial app, I’ve found that you are going to need probably one custom control in order to make the app UI seem polished.

Most (all?) UI frameworks allow the extension/composition/creation of its controls or widgets. Sometimes there is a control that is close enough to what you want that you can subclass it, but many times you end up drawing it yourself using some combination of a low-level drawing API (GDI+ anyone?) and image files.

Further, there is the issue of which UI framework to use. Cross platform development and native look-and-feel are at odds with one another, and some cross-platform frameworks (Swing comes to mind) just wrap the native UI elements for the platform. This is good for getting the app to look like a native app on a platform, but unfortunately it represents kind of an uncanny valley of look-and-feel. Yes, it uses native widgets, but something just seems a little bit off.

Taking things further, we might want to think about mobile devices. QT and WXWidgets are available on mobile devices also, and probably many others. Each mobile platform also has a native set of widgets. How do you decide which way to go with the UI?

Lately I’ve started just punting when it comes to deciding anything native. I’m usually doing most things as JS/HTML and then just loading it in a web widget, which most platforms have. However I’m seeing things like QT’s QML, which is a Javascript-based widget modeling language. This brings us around full-circle on Web apps vs native apps. For apps that don’t require any custom widgetry, I supposed you could just duplicate the UI for many simpler apps. But once you have some investment in custom controls, does it make sense to just use HTML and share it between platforms?

I’m not too sure about this yet. I think I’m going to abandon the idea of using any native UI toolkit for my outliner experiment.

Written by newcome

June 11, 2012 at 12:35 pm

Posted in Uncategorized