A couple of days ago, I shared a new version of the Employee Directory sample application. By default, the application uses an in-memory data store to provide a “download and run” experience (no need to set up a server or a database). In this post, I’ll show how to replace the in-memory data store with a database hosted on Parse.com.

Step 1: Sign Up for Parse.com

Go to http://parse.com and sign up for Parse.

Step 2: Create an Application

On the next screen, create an application as follows:

Step 3: Get your Application Keys

In the dashboard, go to Settings and grab the Application ID and the JavaScript key.

Step 4: Populate the Database

There are two ways to populate the database:

1. Import a .json, .csv, .xls, or .txt file from the Data Browser interface. This approach works well for simple data structures.
2. Programmatically, using the the Parse SDK.

For the employee directory application I wrote the quick script below to populate the database:

Step 5: Run the Employee Directory App using the Parse.com Cloud Data

1. Download the source code for the Employee Directory client application available in this repository.
2. Extract the application files to a directory you can reach using a local or hosted web server.
   
   Because the templates of this application are loaded using XMLHTTPRequest, you will get a cross domain error (Access-Control-Allow-Origin) if you load index.html from the file system (with the file:// protocol). Make sure you load the application from a web server. For example: http://localhost/directory-backbone-bootstrap.
   
3. Open index.html in your code editor.
4. Comment out the “model-in-memory.js” script, and uncomment the “parse-1.2.2.min.js” and “model-parse-com.js” scripts.
5. Open index.html in a Browser and test the application.

Step 6: Secure the Database

Parse.com offers different security options. A complete discussion of these options is beyond the scope of this article. Since Employee Directory is a read-only and public application, we will simply:

1. Prevent client class creation (In Settings > General Settings):

2. Disable create, update, delete operations (In Data Browser > More > Set Permissions):

Step 1: Install Node.js

If it’s not already installed on your system, follow the steps below to install Node.js:

1. Go to http://nodejs.org, and click the Install button.
2. Run the installer that you just downloaded. When the installer completes, a message indicates that Node was installed at /usr/local/bin/node and npm was installed at /usr/local/bin/npm. At this point node.js is ready to use.

Step 2: Install MongoDB

If MongoDB is not already installed on your system, refer to the MongoDB QuickStart for platform-specific installation instructions. Here are some quick steps to install MongoDB on a Mac:

1. Go to http://www.mongodb.org/downloads and download the latest OS X 64-bit production release (2.4.2 at the time of this writing).
2. In Finder, double-click mongodb-osx-x86_64-2.4.2.tgz to extract its contents. (Alternatively you can use: tar -zxvf mongodb-osx-x86_64-2.4.2.tgz)
3. Move the mongodb-osx-x86_64-2.4.2 folder to /usr/local (or another folder according to your personal preferences):

   sudo mv -n mongodb-osx-x86_64-2.4.2/ /usr/local/
   

4. (Optional) Create a symbolic link to make it easier to access:

   sudo ln -s /usr/local/mongodb-osx-x86_64-2.4.2 /usr/local/mongodb
   

5. Create a folder for MongoDB’s data and set the appropriate permissions:

   sudo mkdir -p /data/db
   sudo chown `id -u` /data/db
   

6. Start mongodb

   cd /usr/local/mongodb
   ./bin/mongod
   

Step 3: Download the Code

The Node.js code for the Employee Directory RESTful services is available in this GitHub repository. Download it here and unzip the file anywhere on your file system.

Open server.js and routes/employee.js in a code editor. The RESTFul API is defined as follows:

Step 4: Install dependencies (Express Framework and MongoDB driver)

Open package.json in a code editor and note that the application has two dependencies: the Express framework and the MongoDB driver. To install these dependencies, open a command line (terminal window), cd to the directory where you extracted the source code, and type:

npm install

Step 5: Start the server

Make sure MongoDB is started (Step 2 point 6 above) before you start your Node.js server as follows:

node server.js

Step 6: Testing the API in your Browser

• Retrieve all employees

  http://localhost:3000/employees

• Retrieve all employees whose name includes the letter “j”

  http://localhost:3000/employees?name=j

• Retrieve employee with id == 1

  http://localhost:3000/employees/1

• Retrieve employees that are direct reports of employee number 1

  http://localhost:3000/employees/1/reports

Step 7: Test the API with the Employee Directory Client Application

1. Download the source code for the Employee Directory client application available in this repository.
2. Unzip the file anywhere on your file system.
3. Open index.html in your code editor.
4. Comment out the “model-in-memory.js” script, and uncomment the “model-jsonp.js” script to use the JSONP adapter that connects to your Node.js server.
5. Make sure the MongoDB and Node.js servers are started
6. Open index.html in a Browser and test the application

Related Post

Check out this post for another Node.js example with a more complete RESTful API including the ability to add, modify and delete items (POST, PUT, and DELETE HTTP methods).

A year ago, I blogged Employee Directory, a sample application that demonstrates how to build modern web apps with Backbone.js and Twitter Bootstrap. With 1200+ stars and 300+ forks on GitHub, it has been my most popular sample application to date.

Because of the continued interest in the application as a starting point and a reference for Backbone.js and Twitter Bootstrap, I decided to give it a well deserved update.

New in this version:

• Latest version of Backbone.js (1.0)
• Latest version of Twitter Bootstrap (2.3.1)
• Layout is now responsive
• Data persistence layer is now abstracted (see below)
• Additional data persistence options: In-memory datastore, Node.js/MongoDB, Parse.com (In addition to PHP and Java)
• New GitHub repository organization (see below)
• Bug fixes and cleaner code

Running the Application

You can test the hosted version of the application here, or you can download the code (see GitHub repository information below) and test the application locally.

Data Persistence Abstraction

Among the great feedback I received, a number of you mentioned you wanted a self-contained version of the app with no dependency on a specific back end to be able to “download and run” without having to set up a server and a database. So I rearchitected the persistence layer with simple and pluggable data adapters. By default the application now uses an in-memory data store, but other adapters are available. To change the data persistence strategy, just comment out model-in-memory.js in index.html and uncomment one of the other data adapters using the table below as a reference.

I initially published the application with two implementations of a RESTful back end: A Java version using Jersey, and a PHP version using Slim. A year later, I’m adding the long overdue Node.js/MongoDB back end.

The Node.js, Java, and PHP RESTful back ends work with both JSON and JSONP. If the request includes a query parameter named “callback”, a JSONP response is returned, otherwise, a regular JSON response is returned.

New GitHub Repositories

The original repository (backbone-directory) grew somewhat chaotically as I kept adding different versions and different data persistence options. So I decided to start fresh with new repositories.

The client code for the application is available in the new directory-backbone-bootstrap repository. In the future, this new repository naming scheme will allow me to add different versions of the app (for example, directory-backbone-foundation or directory-angular-bootstrap) while keeping each repository clean and focused.

If you want to go beyond the in-memory datastore, you can download the REST services in the following repositories:

directory-rest-nodejs (Node.js/MongoDB implementation)
directory-rest-php (PHP implementation)
directory-rest-java (Java implementation coming soon)

I’ve recently been working on a few applications that integrate with Facebook. As a reference for myself and other developers building similar applications, I thought it would be useful to put together a sample application that demonstrates some commonly used integration scenarios and workflows when using the Facebook JavaScript SDK and the Graph API. So, Sociogram is a sample application that demonstrates how to:

• Load the Facebook SDK
• Login with Facebook
• Request specific permissions
• Revoke permissions
• Get data (list of friends, mutual friends, feed items, etc.)
• Post to the wall using your application’s custom UI or the Facebook dialogs
• Listen to Facebook events such as status change, etc

The Graph API provides access to a lot more data and actions, but you are essentially always using the same API with a different path into the graph.

There are other examples out there, each of them built around different use cases. This application is built with Backbone.js using a single page architecture, but the code can easily be reused in a different context.

Running the Application

Click here to run the application in your browser.

Source Code

The source code is available in this GitHub repository.

Mobile App Version

In my next post, I will share a PhoneGap version of the same application (using the PhoneGap Facebook plugin).

Disclaimer

This is a sample application, not a production application. Some trade-offs were made to keep the code generic, simple and readable.

In my previous post, I shared my recent experience building a RESTful API with Node.js, MongoDB, and Express.

In this post, I’m sharing the client application that uses that RESTful API. The Node Cellar application allows you to manage (retrieve, create, update, delete) the wines in a wine cellar database.

The client application is built with Backbone.js and Twitter Bootstrap.

Run the Application

You can run the application here. For obvious reasons, the create, update, delete features have been disabled in this hosted version.

NOTE: Node.js is running on port 3000 on my EC2 Instance. My friend James Ward convinced me to host the application on their service over at Heroku. It’s now running on port 80 at http://nodecellar.coenraets.org. Thanks James!

Server-Side

The details of the Node.js, MongoDB, and Express implementation are documented in my previous post.

Client-Side

In this application, Node.js is used to provide the RESTful services that the client application needs to manipulate the data. Node.js is not used to generate HTML/Views at the server-side. Instead, the Views are created dynamically at the client-side using Backbone.js and Underscore.js templates. They are injected into- and removed from the DOM as needed as you navigate through the application. Node Cellar is a “single page application”.

This is a Node.js and MongoDB powered version of an application I initially posted here with PHP and Java backends. The original post provides some additional Backbone.js context. This Node.js version also provides an improved user experience with the latest version of Twitter Bootstrap (2.1.1), and a responsive layout. It also uses Adobe Edge Web Fonts.

Source Code

The source code is available in this repository on GitHub.

You can watch a short video below. In particular, at the end of the video, you will see how to reset the local database (to get rid of the sample dataset), and synchronize with your own data. Go full screen for a better viewing experience.

The application was built with Backbone.js and PhoneGap.

Synchronizing with your own data

Employee Directory always accesses the employee data from a local database on your iPhone. The information is always available, even when you are offline. The application comes with a sample dataset to provide an “out-of-the-box” experience, and with a simple offline synchronization mechanism to sync with your own data.

I created a companion microsite for the application (http://employeedirectory.org/app) that provides the steps to synchronize with your own data.

The offline synchronization strategy used in the application is documented in a previous post: Simple Offline Data Synchronization for Mobile Web and PhoneGap Applications.

User Interface

The styling and skinning of the application is documented in another post: Crafting Native Looking iOS Apps with HTML, Backbone.js, and PhoneGap.

Source Code

I’m also making the source code for this application available in this repository on GitHub.

In this article, I describe a simple data synchronization strategy that uses the device’s (or browser’s) SQLite database. The implementation currently leverages the Web SQL API (even though the W3C is no longer actively maintaining the spec) because both iOS and Android support it, but they don’t support IndexedDB, the official alternative. However, the API described below — getLastSync(), getChanges(), applyChanges() — defines a generic synchronization contract, and the solution can be expanded and made “pluggable”: You could create different synchronization objects, each providing a different implementation of these methods. You could then choose which object to plug in based on the context and the platform your application is running on.

Try it in the Playground

Before looking at the code, you can try some offline syncing in this a hosted playground:

1. Open the Offline Client Playground in Chrome or Safari (they both support Web SQL).
2. Click the Synchronize button.
3. Look at the log (the textarea in the middle of the screen): Because it’s the first time you use the application, all the employees have been downloaded from the server and inserted in your local SQLite database.
4. Clear the log, click the Synchronize button, and look at the log again: because you now have an up-to-date local version of the data, the server didn’t return any change and your local database remains unchanged.
5. In another tab, open the Server Admin PlayGround.
6. Modify an existing employee and click Save. (Don’t worry, it’s using your own session-based data set).
7. Go back to the Offline Client tab, click Synchronize, and notice that the server returned one change, and that it was applied to your local database.
8. Go back to the Server Admin tab and modify (create, update, delete) other employees. Switch back to the Offline Client tab, click Synchronize, and see how these changes are applied to your local database.
9. You can also use the Resources Tab in the Chrome Developer Tools to inspect your local database.

Server API

The only piece of infrastructure you need at the server side is an API that returns the items that have changed (created, updated, or deleted) since a specific moment in time expressed as a timestamp.

Here is the RESTful API call used in my application:

http://coenraets.org/offline-sync/api/employees?modifiedSince=2012-03-01 10:20:56

The format of the data returned by the server is up to you and is part of the contract between the client and the server. In this application, the server returns the changes as an array of JSON objects. The server-side technology (RoR, PHP, Java, .NET, …) and database system (SQL, NoSQL, …) you use to generate the list of changes is also totally up to you. I provide a simple PHP implementation as part of the source code. That implementation manages a session-based data set that provides an isolated and transient playground. In a real-life application, you’d obviously get the data from some sort of database.

Client API

At the client side, our synchronization API consists of three methods.

getLastSync()

A method that returns a timestamp to be used as the query parameter for the next synchronization request. A common practice is to persist a timestamp after each synchronization request. But things can go wrong and the timestamp itself can get out-of-sync. I prefer to “recalculate” the lastSync timestamp before each synchronization request.

getLastSync: function(callback) {
    this.db.transaction(
        function(tx) {
            var sql = "SELECT MAX(lastModified) as lastSync FROM employee";
            tx.executeSql(sql, this.txErrorHandler,
                function(tx, results) {
                    var lastSync = results.rows.item(0).lastSync;
                    callback(lastSync);
                }
            );
        }
    );
}

getChanges()

This is a wrapper around an Ajax call to the server-side API that returns the items that have changed (created, updated, or deleted) since a specific moment in time defined in the modifiedSince parameter.

getChanges: function(syncURL, modifiedSince, callback) {

    $.ajax({
        url: syncURL,
        data: {modifiedSince: modifiedSince},
        dataType:"json",
        success:function (changes) {
            callback(changes);
        },
        error: function(model, response) {
            alert(response.responseText);
        }
    });

}

applyChanges()

A method that persists the changes in your local data store. Notice that SQLite supports a convenient “INSERT OR REPLACE” statement so that you don’t have to determine if you are dealing with a new or existing employee before persisting it.

applyChanges: function(employees, callback) {
    this.db.transaction(
        function(tx) {
            var l = employees.length;
            var sql =
                "INSERT OR REPLACE INTO employee (id, firstName, lastName, title, officePhone, deleted, lastModified) " +
                "VALUES (?, ?, ?, ?, ?, ?, ?)";
            var e;
            for (var i = 0; i < l; i++) {
                e = employees[i];
                var params = [e.id, e.firstName, e.lastName, e.title, e.officePhone, e.deleted, e.lastModified];
                tx.executeSql(sql, params);
            }
        },
        this.txErrorHandler,
        function(tx) {
            callback();
        }
    );
}

Synchronization Logic

With these server and client APIs in place, you can choreograph a data synchronization process as follows:

sync: function(syncURL, callback) {

    var self = this;
    this.getLastSync(function(lastSync){
        self.getChanges(syncURL, lastSync,
            function (changes) {
                self.applyChanges(changes, callback);
            }
        );
    });

}

Final Notes

• This solution currently supports unidirectional (server to client) data synchronization. It could easily be expanded to support bidirectional synchronization.
• This solution currently implements “logical deletes”: items are not physically deleted from the table, but the value of their “deleted” column is set to true.
• As mentioned above, you could replace the Web SQL implementation with another data access strategy. For example, take a look at Brian Leroux’ Lawnchair for another local persistence solution.

Source Code

The source code is available in this GitHub repository.

To demonstrate these features, I decided to revisit my wine cellar application, which was in need of a serious UI makeover.

You can run the application here.

This online version uses an in-memory datastore: all your changes will be lost the next time you start the application or hit your browser’s refresh button. The image upload feature is also disabled, but you can still drag an image from your file system and drop it in the Wine Form, which is pretty cool (see the note about browser support below). The source code available on GitHub includes a persistent back-end (in addition to the in-memory datastore), and a fully functional implementation of the file upload feature.

Code Highlights

Template Loader

This application features a template loader that now uses jQuery “deferreds” to load the HTML templates more efficiently.

Drag-and-Drop File Upload and HTML 5 File API

To upload an image file for a Wine, drag a file from your file system and drop it in the image box inside the Wine Form: The image is automatically displayed inside the img tag. This is done using the HTML 5 File API and doesn’t require a server roundtrip.

NOTE: This feature only works in Chrome at this time. I will implement an alternative implementation in a future version of the application to provide a consistent behavior across browsers.

The image is uploaded to the server using Ajax (XHR) when you save the form: no page refresh or iframe hack.

Paging

Twitter Bootstrap provides easy markup and styles to create paginated lists. In this application, the Bootstrap markup and styles are “componentized” or “widgetized” into a Backbone View (Paginator), which adds the appropriate pagination behavior.

NOTE: In the current implementation of this application, the entire data set is always retrieved from the server and paging is provided for a cosmetic/layout reason. You could easily replace this implementation with a true paging strategy where pages are lazy-loaded from the server as needed.

Forms

Backbone Cellar also uses Twitter Bootstrap’s forms, which greatly help with form layouts (see WineView).

Validation

Twitter Bootstrap also provides simple markup and styles to highlight validation errors in forms. In Backbone Cellar, Bootstrap’s validation markup and styles are wired with validation rules defined in the Backbone model. Note that at this time, the application doesn’t use the Backbone model’s default validate() method, but custom validateItem() and validateAll() methods instead.

The application also uses other Twitter Bootstrap features including thumbnails, dropdowns, alerts, etc.

Source Code

The source code is available in the bootstrap folder of the backbone-cellar repository on GitHub.

Disclaimer

This is a sample application, not a production application. Some trade-offs were made to keep the code generic, simple and readable. In a real-life application, you should consider implementing a namespacing scheme to keep the global namespace clean, and other optimization techniques such as view and/or data caching.

After my recent post, Crafting Native Looking iOS Apps with HTML, a number of you asked for an offline version that would use a Local Database (instead of the simple in-memory store) and provide a mechanism to automatically keep the local database in sync with a server database.

I’ll save automatic data synchronization strategies for a future post, but here is the first step: an “offline” version of the application that uses the device’s or the browser’s local database as its data provider. This version still uses Backbone.js as its architectural framework. Backbone.js makes it easy to change its default data access mechanism (which assumes RESTful services). You just replace the default Backbone.sync implementation and provide your own data access logic: in this case, some local SQL logic.

Web SQL vs IndexedDB

As you probably know, there have been two competing database APIs for HTML. From the W3C web site:

• The Web SQL specification defines an API for storing data in databases that can be queried using a variant of SQL. This specification is no longer in active maintenance and the Web Applications Working Group does not intend to maintain it further.
• The Indexed Database specification defines APIs for a database of records holding simple values and hierarchical objects. It is a working draft, and “work in progress”.

Even though the W3C is no longer actively maintaining the spec, this application uses the Web SQL API because, as a mobile app, its two main target platforms are iOS and Android, which both currently support Web SQL but not IndexedDB. More detailed platform support information can be found on caniuse.com (Web SQL and IndexedDB).

Chrome, Safari, and Opera on the desktop also support Web SQL, which means that you can run the application in these browsers. Try it here. For example, using the Chrome Developer Tools you could debug the application and inspect the database as shown in this screenshot:

Firefox and IE don’t support Web SQL. You could easily create an alternative version of EmployeeDAO (described below) that uses IndexedDB instead. You could also create a version of the application that uses either Web SQL or IndexedDB depending on the platform it’s running on.

Code Highlights

The source code is available in the localdb folder of the backbone-directory repository on GitHub. Here is a quick walkthrough…

The data access logic is encapsulated in EmployeeDAO, which also has a “populate” function to populate the employee table with sample data.

directory.dao.EmployeeDAO = function(db) {
    this.db = db;
};

_.extend(directory.dao.EmployeeDAO.prototype, {

    findByName: function(key, callback) {
        this.db.transaction(
            function(tx) {

                var sql = "SELECT e.id, e.firstName, e.lastName, e.title, count(r.id) reportCount " +
                    "FROM employee e LEFT JOIN employee r ON r.managerId = e.id " +
                    "WHERE e.firstName || ' ' || e.lastName LIKE ? " +
                    "GROUP BY e.id ORDER BY e.lastName, e.firstName";

                tx.executeSql(sql, ['%' + key + '%'], function(tx, results) {
                    var len = results.rows.length,
                        employees = [],
                        i = 0;
                    for (; i < len; i = i + 1) {
                        employees[i] = results.rows.item(i);
                    }
                    callback(employees);
                });
            },
            function(tx, error) {
                alert("Transaction Error: " + error);
            }
        );
    },

    findById: function(id, callback) {
        // removed for brevity
    },

    findByManager: function(managerId, callback) {
        // removed for brevity
    },

    populate: function(callback) {
        // removed for brevity
    }
});

Models are annotated with a “dao” attribute to indicate which data object to use to access their underlying data.

directory.models.Employee = Backbone.Model.extend({

    dao: directory.dao.EmployeeDAO,

});

directory.models.EmployeeCollection = Backbone.Collection.extend({

    dao: directory.dao.EmployeeDAO,

    model: directory.models.Employee,

});

With that infrastructure in place, you can then override Backbone.sync to access data from the local database instead of RESTful services:

Backbone.sync = function(method, model, options) {

    var dao = new model.dao(directory.db);

    if (method === "read") {
        if (model.id) {
            dao.findById(model.id, function(data) {
                options.success(data);
            });
        } else if (model.managerId) {
            dao.findByManager(model.managerId, function(data) {
                options.success(data);
            });
        } 
        // removed for brevity
    }

};

Source Code

The source code is available in the localdb folder of the backbone-directory repository on GitHub.

I’ve been blogging a lot about Backbone.js recently. Backbone.js is a lightweight architectural framework that brings structure to your Web applications. Backbone is not, however, a user interface framework that helps you with the way your application looks.

So, where do you turn to for help when you need to make your application look good?

For traditional web apps (delivered through a browser), Twitter Bootstrap can help (read here). But what about Mobile apps? I explored Backbone.js + jQuery Mobile here. Depending on what you are looking for, it may or may not be the right solution: jQM provides mobile skins, but they don’t look native. It’s also more of a full stack framework than a lightweight UI toolkit that you can easily layer on top of your app.

The alternative to using an existing UI toolkit is to roll your own styles to make your application look and behave like a native app. Sounds easy enough, but when you consider all the details and want to achieve “pixel perfection”, it becomes a daunting task.

As I was getting ready to tackle the challenge, and build a new native looking version of my Employee Directory app, I came across this great blog post by Chee Aun where he documents the process he went through to build his own Hacker News mobile app. His post is a real gem, and I ended up reusing a lot of the Hacker News app styles.

Compared to the Hacker News app, the Employee Directory page flow is more random. Here are a few examples:

1. SearchPage -> EmployeePage -> ReportsPage -> EmployeePage -> …
2. SearchPage -> EmployeePage -> EmployeePage (manager) -> Reports -> …
3. SearchPage -> EmployeePage -> EmployeePage (manager) -> EmployeePage (manager’s manager) -> …

As you can see, the page flow includes “same page transitions”, when the user navigates from one employee to his/her manager. To accommodate the Employee Directory page flow requirements, my Backbone.js infrastructure creates and destroys pages as needed with the appropriate slide-in/slide-out transitions. The implementation of these transitions was inspired by Wesley Hales’ article.

PhoneGap

Even though you can run this application in a browser (here), I built it with the intention of packaging it as a native app with PhoneGap so that you could start it like any other app from your iPhone home screen. If you are not familiar with PhoneGap, I’ll provide more details on packaging this app as a native app in my next post.

Source Code

I updated the backbone-directory GitHub repository to include this version: It is available in the iphone directory.