nodePDX 2012
Table of Contents
- Background
- Day 1
- 9:30am Day 1 Kickoff, by Troy Howard and Adron Hall
- 10:00am A CouchDB, Neo4j, Redis, and Node.JS Circus, by Eric Redmond
- 11:00am Building Applications with Bricks.js, by Jerry Sievert
- 11:45am Lunch Break
- 1:00pm Object-Oriented Patterns in JavaScript, by Jesse Hallett
- 2:00pm Missing from the Beginning: The Federation of Wikis, by Ward Cunningham
- 3:00pm ql.io - Consuming HTTP APIs at Scale, by Subbu Allamaraju
- 4:00pm NODE! Huh! What else is it good for?, by Scott Koon
- 4:45pm Hackathon and Unconference
- 5:45pm To the Bars! Cleanup and GTFO., by Troy Howard and Adron Hall
- 6:00pm Happy Hour @ The Lotus
- Day 2
- 9:00am Meet n' Greet
- 9:30am Day 2 Kickoff, by Troy Howard and Adron Hall
- 10:00am Building a Real-time geolocation game with Geoloqi using Node.JS, by Kyle Drake
- 11:00am BattleBots in the Clouds with Node.js, by Ken Robertson
- 11:45am Lunch Break
- 1:00pm DRYing out your client-side apps, by Garann Means
- 2:00pm Building Node.JS Applications on Windows Azure, by Aaron
- 3:00pm Geddy - A better framework for building apps on node, by Daniel Erickson
- 4:00pm Better Together: Building Scalable Real Time Collaborative Apps with Node.js, by Kav Latiolais
- Persistent on client and server
- Best practice: never change model
- Node server gets a message
- Controller layer is always on the node server
- Client changes don't persist (localStorage) with model changes
- Could mark the model as dirty during the callback to wait for Socket.io
- Be very careful when not using set() to replace server-provided models
- Auth that has shared client and server
- Push full text rather than addressing a diff problem
- Creating real time applications have same network effect as social
- 4:45pm Hackathon and Unconference
- 5:45pm To the Bars! Cleanup and GTFO., by Troy Howard and Adron Hall
- 6:00pm After Party @ The Guild
Background
Day 1
9:30am Day 1 Kickoff, by Troy Howard and Adron Hall
AppFog
New Relic
10:00am A CouchDB, Neo4j, Redis, and Node.JS Circus, by Eric Redmond
Eric Redmond @coderoshi NoSQL databases are fun, and we've all wired up a Node.js project to use one. But what about two? What about three?
That's what's on the docket for this talk. We'll do all the things:
- Walk through a (very) quick intro on CouchDB, Neo4j, and Redis.
- Then learn how to interact with these very different databases using Node.js.
- Then wire them up into a single web application (using Bricks.js, for good measure)
- Write it all in CoffeeScript… buzzword overload!!!
If you do not walk out of this talk excited and maybe a little confused, I will have failed.
Book: Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement
Systems of record
Always define the gold standard.
In this application the CouchDB is the system of record.
CouchDB
% couchdb Apache CouchDB 1.1.0 (LogLevel=info) is starting. Apache CouchDB has started. Time to relax. [info] [<0.31.0>] Apache CouchDB has started on http://127.0.0.1:5984/
Neo4j
Has a Gremlin query support.
Good at managing connections between objects.
Redis
As a key-value store good for autosuggest and documents and data transformation from flat files.
CoffeeScript
Notes
This implementation is a "polyglot" system: uses Redis for pre-transformation.
Setup
Have to have each of the databases running for the application to work.
Run the import of the data
This is the prepopulate.coffee
jwalsh@jwalshmac1:~/sandbox/NCNRCSBuzzSpec master* % coffee pre_populate.coffee Lines Processed: 1000 Lines Processed: 2000 Lines Processed: 3000 Lines Processed: 4000
Redis
redis 127.0.0.1:6379> keys band:* redis 127.0.0.1:6379> smembers band:Bliss 1) "Dave Foster" 2) "Kurt Cobain"
CouchDB
http://localhost:5984/_utils/document.html?bands/Nirvana
jwalsh@jwalshmac1:~/sandbox/NCNRCSBuzzSpec master* % coffee populate_couch.coffee Bands Loaded: 1000 Bands Loaded: 2000 Bands Loaded: 3000
The data for this can also be seen with
Modules
init() if (!module.export)
Populating and querying Neo4j
Need to look at the work that was associated with this project rather than trying to use the client on NPM.
The population is run with
jwalsh@jwalshmac1:~/sandbox/NCNRCSBuzzSpec master* % coffee graph_sync.coffee
{ doc: 8260,
band: 0,
artist: 0,
member: 0,
role: 0,
The output from the
http://docs.neo4j.org/chunked/stable/tools-webadmin.html
http://127.0.0.1:7474/webadmin/
http://docs.neo4j.org/chunked/stable/cypher-plugin.html
Using that interface the following can be used to query with Cypher:
cypher> start band=node(1) match band-[:member]->artist return band, artist
cypher>
==> +-----------------------------------------------------------------------------+
==> | band | artist |
==> +-----------------------------------------------------------------------------+
==> | Node[1]{name->"New Power Generation"} | Node[218]{name->"Tommy Barbarella"} |
==> | Node[1]{name->"New Power Generation"} | Node[219]{name->"Prince"} |
==> | Node[1]{name->"New Power Generation"} | Node[217]{name->"Maceo Parker"} |
==> | Node[1]{name->"New Power Generation"} | Node[215]{name->"Mayte Garcia"} |
==> | Node[1]{name->"New Power Generation"} | Node[214]{name->"Levi Seacer, Jr."} |
==> +-----------------------------------------------------------------------------+
==> 5 rows, 4 ms
11:00am Building Applications with Bricks.js, by Jerry Sievert
Jerry Sievert @jerrysievert Bricks.js is fast modular framework built on node.js. This session will be a mix of an introduction for those who have not used it, and building a fairly simple application using it.
Links
Bricks.js: http://bricksjs.com/ Blog: http://legitimatesounding.com/blog/ Twitter: @jerrysievert Github: https://github.com/JerrySievert
Dailly Insight Group
Apache routing
Have varying levels of hooks that exist for routes. This is the same metaphor.
- Pre = ACL
- Main
- Post = cleanup
- Final = accounting
Plugins
exports = {
init: function() {},
plugin: function() {},
end: function() {},
}
Dependencies
Presenter normally used vows for unit testing of the functionality.
See the approximate versioning in ~/sandbox/nodepdx-bricksjs/project/:
"dependencies": { "bricks": ">=1.1.1", "marked": ">=0.1.9", "mongoskin": "~0.3.0", "handlebars": ">=1.0.0"
Settings
See the invocation provided for having default routes defined in 0.6.*:
var settings = require('./settings.json');
11:45am Lunch Break
1:00pm Object-Oriented Patterns in JavaScript, by Jesse Hallett
Jesse Hallett @hallettj JavaScript is a language that is tremendously flexible, but that comes with few clear conventions. Code organization is one are that has been reinvented perhaps more times than there are JavaScript programmers. I will examine a few tools that are intended to improve the experience of writing object-oriented code.
Some of the specific tools that I plan to examine are traits.js, CoffeeScript, Prototype, and JiverScripts. The goal is to get a look at different ideas to see what each approach has to offer. We will discuss inheritance versus composition, and question when it is or is not appropriate to use object-oriented code.
Audience participation is encouraged: if you have a favorite OO implementation, or experiences with JavaScript code organization, feel free to come and to share.
Composition
Looked at a physics game: composition is flat and based on generic setup profile.
Traits
The presenter provided an implementation of traits in JavaScript to support composition of objects.
https://github.com/hallettj/etiquette.js
See also,
Protocols
All one has to implement for map(), reduce(), and filter() is a forEach(). This is based on protocols in Clojure:
2:00pm Missing from the Beginning: The Federation of Wikis, by Ward Cunningham
Ward Cunningham @WardCunningham Our new wiki innovates three ways. It shares through federation, composes by refactoring and wraps data with visualization.
The Smallest Federated Wiki project wants to be small in the "easy to learn powerful ideas" version of small. It wants to be a wiki so that strangers can meet and create works of value together. And it wants to be federated so that the burden of maintaining long-lasting content is shared among those who care.
3:00pm ql.io - Consuming HTTP APIs at Scale, by Subbu Allamaraju
Subbu Allamaraju @sallamar Node.js is a great platform for building I/O bound apps. At eBay, my team applied node.js to solve a very common chore - how to get data from server-side HTTP APIs (or "web services") quickly. ql.io is a result of this work.
ql.io consists of two parts:
- A SQL + JSON inspired DSL for HTTP
- A runtime that you can can either deploy as an HTTP gateway or use as a JS API for node.js based apps.
In this talk, I will show how you can use ql.io, the agility and performance gains that ql.io can bring in, and then take a deep dive into some of the design choices we made under the hood.
Links
- Blog: http://subbu.org
- Project site: http://ql.io
- Twitter: @sallamar
- Company: http://ebay.com/
- Github: https://github.com/ql-io/ql.io and https://github.com/s3u
- http://www.slideshare.net/sallamar/qlio-at-nodepdx
Consume and large scale APIs
Proxy and store the results of API calls
4:00pm NODE! Huh! What else is it good for?, by Scott Koon
Scott Koon @lazycoder Everyone talks about Node.js in terms of non-blocking I/O and creating a web service or web site using JavaScript. But there is more to Node.js than just Sockets, ports, and protocols. I'll explore some of the non-web exclusive uses of Node.js. Node can be a code compiler, an FTP server, a continuous integration server, a mail server, a deployment server, or an IRC server. Node can provide system reports, build and package your projects, and parse ePub books.
Links
- Blog : http://lazycoder.com/
- Twitter : @lazycoder @LazycoderLinks
- GitHub : http://github.com/skoon
- BitBucket : http://bitbucket.org/scott_koon
- podcast : http://herdingcode.com
Concrete
CI server; not fully featured.
ePub
Could be useful for prototyping the Redis / Neo4j work.
4:45pm Hackathon and Unconference
5:45pm To the Bars! Cleanup and GTFO., by Troy Howard and Adron Hall
6:00pm Happy Hour @ The Lotus
Day 2
9:00am Meet n' Greet
9:30am Day 2 Kickoff, by Troy Howard and Adron Hall
10:00am Building a Real-time geolocation game with Geoloqi using Node.JS, by Kyle Drake
Kyle Drake @kyledrake There are very powerful things you can do with Node.JS, particularly with projects needing a lot of I/O operations. At Geoloqi, we have used Node.JS and Socket.IO to build a JavaScript client that allows our developers to map real-time tracking on a browser with almost no code needed. Our first project using this is MapAttack!, a truly real-time location-based geofencing game.
Geoloqi
Based on where a user is provide useful feedback:
- where are my friends
- location based notes
Used in http://spotsi.com/ .
Implementation details
- iPhone
- Redis pub/sub for immediate updates
- Persistent store eventually gets the data
- socket.io for long polling
- Message queues (http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend/)
Reactor patterns in Node
Performance: Persistent storage
- EBS
- Oracle
Some review of options for sharding and consistency:
Hardware: SSD
Brewer's CAP
Don't use map/reduce as a DB.
Map Attack for nodePDX
- Simple location with radius, no location topology support https://github.com/bjornharrtell/jsts
11:00am BattleBots in the Clouds with Node.js, by Ken Robertson
Ken Robertson @krobertson Years ago I always enjoyed the BattleBots show on TV and loved teams' ingenuity and inspiration to essentially build something only to have it gloriously destroyed. But is there a way we can bring that to the software world? What if we could leverage the cloud to orchestrate scalable battles.
11:45am Lunch Break
1:00pm DRYing out your client-side apps, by Garann Means
Garann Means @garannm There's plenty of cool stuff Node offers purely in terms of server-side architectures, but it also offers a way to solve a problem we've been wrestling with since client-side applications became a big deal: writing everything twice. Rather than having the templates that produce markup exist in one backend language and in JavaScript, you can reuse them. Instead of validating in JavaScript on the client for the user's convenience and then again in some other language on the server for security, you can share a validation module that can be used in both scenarios. And so on. We'll look at some of the ways to stop repeating ourselves in Node apps and focus on getting the most out of existing client-side code.
Focus on the client first
Requirements management will still be difficult
Build systems for dependencies still need server build
Tools
2:00pm Building Node.JS Applications on Windows Azure, by Aaron
Aaron @Aaronontheweb Microsoft is moving towards making Windows Azure a PaaS cloud capable of hosting applications of any shape, size, framework, and programming language, and Node.JS is one of the first non-.NET technologies we've made a first-class citizen on Windows Azure.
Change in organizational leadership to make Node a first-class citizen on Azure
Example posting
jwalsh@jwalshmac1:~ % curl –data-urlencode 'logMessage[message]=hellotheworld' http://httplogger.aaronontheweb.com log
Building an basic application
- Install the Azure PowerTools
Core libraries
https://github.com/tjanczuk/iisnode
- Each IIS instance has a single process it controls
- Node applications instrumented with Node Inspector
- Logs are available via HTTP (js/logs/0x)
Node Inspector can hang app (kill with ?kill param)
3:00pm Geddy - A better framework for building apps on node, by Daniel Erickson
Daniel Erickson @TechWraith Many frameworks have been created to allow you to build apps on Node.js - Express, Matador, and Flatiron to name a few. But none of these frameworks are built with development velocity, backwards compatibility, and speed. This is where Geddy steps in. Geddy is a framework built and battle tested by the JS team at Yammer. It's currently running our upload service. During this talk I'll walk you through building a basic web app with geddy, and show you how we used it to build a prototype mobile site for Yammer in less than 12 hours.
Background
Other language web frameworks
Connect
- Buffer issues
Express
- Sinatra-like
- No structure (but has generator)
Flatiron
- Modules: router, template
Geddy
- Generators
- Have to write own model adapters even for resources
- Has content negotiation (JSON (.json), XML (.xml), JSONP (.js?callback=?))
- Bootstrap + jQuery
- Has metrics support (e.g., calltimer) > npm install metrics
% geddy app my-geddy-app
4:00pm Better Together: Building Scalable Real Time Collaborative Apps with Node.js, by Kav Latiolais
Kav Latiolais @kavla If you're not using node to build collaborative real time applications you might as well be using rails. In this talk we'll discuss patterns and pitfalls of synchronous node apps. We'll roll up our sleeves and dig into some code demonstrating patterns that can help you get started building highly interactive applications that sync real time state with Node.js, Socket.io, and Backbone.js. You will leave this talk with insight on how to build synchronous experiences into your applications and avoid some of the pitfalls we've suffered.
Persistent on client and server
Best practice: never change model
Node server gets a message
Controller layer is always on the node server
Client changes don't persist (localStorage) with model changes
This works only for collaborative applications.
Could mark the model as dirty during the callback to wait for Socket.io
Be very careful when not using set() to replace server-provided models
new or attribute replacement can do bad things given the event proxy.
Auth that has shared client and server
Try to keep the UI or the server state but not both since it adds complexity to the underlying application.