Starting Out With Comet (Orbited) Part 3 – The Client

Comet implementations require both server and client side components.  In Part 2 of this series we installed and configured the server side component and then used the example STOMP Test client to test it out and get a feel for what was going on.

In this part of the series we’ll be covering the client side to make our custom interface to the comet server.  To demonstrate we’ll go through the steps in creating an app I call EZChat which is basically a bare bones comet chat client.  The interface will allow you to choose a name and submit messages that will be broadcast to everyone viewing the page in real time.

For it to work you will have to have the configuration of Part 2 setup and your Orbited server running.  For reference I learned most of the client side code in this example by playing with the source of the example STOMP Test client. I recommend taking a look at that source if you need more advanced options or another example.

Includes

At the top of the <head> you first need to include the libraries for the Orbited (/static/Orbited.js) and STOMP (/static/protocols/stomp/stomp.js) client-side implementations.


<script>document.domain=document.domain</script>
 <script src="http://localhost:9000/static/Orbited.js"></script>
 <script>
 Orbited.settings.port = 9000;
 TCPSocket = Orbited.TCPSocket;
 </script>
 <script src="http://localhost:9000/static/protocols/stomp/stomp.js"></script>

 <script src="http://www.json.org/json2.js"></script>

In between the scripts we’ve set up a TCPScocket.  It must be don between the two script includes because the STOMP library needs the socket setup for its execution.  We’ve also specified the Orbited port which is necessary especially if you change the port on which the orbited and stomp javascript files are hosted on (they can both be hosted on port 80 along with your other scripts).

Lastly, we’ve also included a popular JSON library.  Typically I use jquery-json but I’ve kept this tutorial free of javascript frameworks so as not to add unneeded complexity.  If you’d like you can of course switch to whatever JSON library you’re used to; just replace the JSON.stringify and JSON.parse functions with your equivalents.

STOMP Setup

Because we’re dealing with the setup we’ll skip to the bottom of the page and add the following script just before the end tag of the body (</body>).


<script type="text/javascript">
(function() { // set up stomp client.

stomp = new STOMPClient();

 stomp.onconnectedframe = function() {  // Run on initial connection to STOMP (comet) server
 stomp.ready = true;
 // subscribe to channel CHANNEL = "/ezchat/"
 var CHANNEL = '/ezchat/'
 stomp.subscribe(CHANNEL);
 };

 stomp.onmessageframe = function(frame) {  // Executed when a messge is received
 my_receive( JSON.parse(frame.body) );
 };

 // Everything is setup. Start the connection!
stomp.connect(document.domain, 61613); //, 'guest', 'guest');
})();
</script>

My apologies as always for the crappiness of the wordpress syntax parser.   Lets walk through what’s happening.

At the top, we initialize a new STOMPClient object.  The StompClient has the following hooks you can override to trigger your own events.

onopen – Called when the Transport is opened
onclose – Called when the Transport has closed
onerror – Called when the Stomp Client has errored
onerrorframe – Called when there is an error in the message received
onconnectedframe – Called when a the client is fully set up for sending/receiving
onmessageframe – Called when a message is received

The STOMP object also has these functions for connecting and resetting the connections.

reset – Resets the STOMP connection
connect – Connects to the STOMP server
send – Sends the object in the first argument to the channel specified by the second argument

In our simple example only the onconnectedframe and onmessage frames need to be overwritten.

The onconnectedframe function is called when the STOMP server has been connected to and everything is setup for sending and receiving messages.  Inside this function we simply need to subscribe/listen to the CHANNEL we’ve setup for our chat.  For the example I’ve chosen the channel ‘/ezchat/’.  Once subscribed our STOMP client will receive any messages sent to that channel in real time.  You can subscribe to multiple channels if you’d like, and you can make clients with different channels if you’d like to have different chat rooms.  But for this example we’ll just stick with the hard coded ‘/ezchat/’ channel.

The onmesageframe function is called when a message has been received.  It is passed a frame object with the following structure

frame

  • body: “{“name”:”Dave”,”message”:”awesome this is working”}”
  • headers: Object
    • content-encoding: “utf-8″
    • content-length: “51″
    • content-type: “text/plain”
    • destination: “/ezchat/”
    • message-id: “/ezchat/_3″
  • type: “MESSAGE”

where the body holds the information that has been sent.  The STOMP server and client add extra “type” and “headers” objects to communicate between each other.  The extra information can be very useful for more complicated applications but for our simple example we’re only interested in the frame “body”.

So you see that the onmessageframe is simply parsing the json object in frame.body of every received message and passing it to my_receive, a function we will soon create.

The Content

For EZChat we need a form where users can specify a name and type messages to send.  We also need an area to put the messages.  Plop this HTML in at the top of the <body> to handle all of that.

<h2>EZChat - Example Comet Client!</h2>
Everyone viewing this page will see the messsages you submit instantly.
<form id="message_form" action="#">
 Name:
 <input type="text" name="chat_name" id="chat_name"></input>
 Message:
 <textarea name="message" id="message" rows="4" cols="40"></textarea>
 <input type="submit" name="Send" onclick="return my_send(); return false"></input>
</form>
<div id="messages"><!--- All received messages will get placed here ---></div>

There are a few things to notice here.  First, the important elements in the form have ids ‘chat_name’ and ‘message’ and the area that will be containing all the received messages is called ‘messages’.  The names don’t matter except that we’ll use them in the functions we create later.

Second, the onclick event of the submit button is overridden with instead calling the my_send function.  We’ll make this function in the next step.

The Functions

Lastly we need to make the custom my_send and my_receive functions that get called to send and receive messages.  Insert these functions into the head after the includes.

my_send

The my_send function will get the values from the ‘chat_name’ and ‘message’ form elements, combine them in an object, convert the object to json, and then sends it to the ‘/ezchat/’ channel.  The sending is handled using the stomp.send command which takes as input the object to send and second, the channel to send it to.

var CHANNEL = '/ezchat/';
function my_send() { 

// Get the values to send from the form
 var name = document.getElementById('chat_name').value;
 var message = document.getElementById('message').value;

 var msg = {'name': name, 'message': message};

 var json_msg = JSON.stringify(msg);
 stomp.send(json_msg, CHANNEL)
 return false;
 }

my_receive

As discussed earlier the my_receive function gets the JSON parsed version of whatever was sent in ‘frame.body’. In the case of our app its always an object of the format

msg = {‘name’: <some name>, ‘message’: <some message>}

The my_receive function simply takes this object and converts it into a prettier HTML format and appends it to the top of the message list we created in the HTML.

function my_receive( msg ) {
 console.log('received message', msg);
 // append the <msg> to the top of the list of messages.
 var messages_el = document.getElementById('messages');
 var new_message = "
<div><strong>" + msg['name'] + ":</strong> " + msg['message'] + "</div>
";
 messages_el.innerHTML = new_message + messages_el.innerHTML;
 }

That’s it for the code.  Scroll to the bottom of the page for the full version of the source.

Results

Ensure that your Orbited server is running as described in Part 2 of the series.  Then load up the page we’ve made in two or more separate windows.  Choose a different name for each window and start sending each other messages.  You’ll notice that both windows will receive the submitted messages almost instantaneously!  The beauty of Comet!

ezchatIf you open your STOMP Test Client and subscribe to the ‘/ezchat/’ channel you’ll see a more raw input on what’s actually being received by the STOMP clients as you chat.

ezchatstompYou can see how the STOMP Test client is incredibly handy for debugging.

That’s it for this part of the Tutorial!  There are still parts to come including writing a data handler on the server side and hopefully a much requested post on Django integration.

Full Source

For your convenience here’s the full index.html file for this example.


 <html>

<head>
 <script>document.domain=document.domain</script>
 <script src="http://localhost:9000/static/Orbited.js"></script>
 <script>
 Orbited.settings.port = 9000;
 TCPSocket = Orbited.TCPSocket;
 </script>
 <script src="http://localhost:9000/static/protocols/stomp/stomp.js"></script>

 <script src="http://www.json.org/json2.js"></script>

 <script type="text/javascript">
 // These are our custom functions for sending and receiving STOMP messages.
 // They will be sent in the format msg = {'name': somename, 'message': somemessage}

 var CHANNEL = '/ezchat/';

 function my_receive( msg ) {
 console.log('received message', msg);
 // append the <msg> to the top of the list of messages.
 var messages_el = document.getElementById('messages');
 var new_message = "
<div><strong>" + msg['name'] + ":</strong> " + msg['message'] + "</div>
";
 messages_el.innerHTML = new_message + messages_el.innerHTML;
 }
 function my_send() {
 // Get the values to send from the form      

 var name = document.getElementById('chat_name').value;
 var message = document.getElementById('message').value;

 var msg = {'name': name, 'message': message};
 console.log(msg);

 var json_msg = JSON.stringify(msg);
 console.log(json_msg);
 stomp.send(json_msg, CHANNEL)
 return false;
 }       
 </script>

</head>

<body>
<h2>EZChat - Example Comet Client!</h2>
<div>Everyone viewing this page will see the messsages you submit instantly.</div>
<form id="message_form" action="#">
 Name:
 <input type="text" name="chat_name" id="chat_name"></input>
 Message:
 <textarea name="message" id="message" rows="4" cols="40"></textarea>
 <input type="submit" name="Send" onclick="return my_send(); return false"></input>
</form>
<div id="messages"><!--- All received messages will get placed here ---></div>
<script type="text/javascript">
(function() { // set up stomp client.
 stomp = new STOMPClient();
 stomp.onconnectedframe = function() {  // Run on initial connection to STOMP (comet) server
 stomp.ready = true;
 // subscribe to channel CHANNEL = "/ezchat/"
 stomp.subscribe(CHANNEL);          
 };

 stomp.onmessageframe = function(frame) {  // Executed when a messge is received
 console.log('frame is', frame);
 my_receive( JSON.parse(frame.body) );
 };

 // Everything is setup. Start the connection!
 stomp.connect(document.domain, 61613); //, 'guest', 'guest');
})();
</script>

</body>

</html>


As usual, if there are corrections or questions please be sure to leave them in the comments. And remember to subscribe to catch the rest of the series.

Appengine Example: Quick Thoughts

I’ve made a few apps on Google’s Appengine now and am getting to the point where I can pump them out fairly quickly.  I really love that they make user authentication (my least favorite part of web applications) incredibly simplified.

I wrote this app in less than an hour as a simple tool for myself and to test out the authentication tools which I hadn’t gotten a chance to use yet.  I’m sharing the source here in case its of any use to others.

Description

The app is called Quick Thoughts and its a very simple private micro blog (a private twitter).  Basically you log in and can record quick notes to yourself.  They’re dated and only you can see them.

Its all on one page.  You can get a good idea of what it is from this screen shot.  You can also just try it out yourself by logging into: http://quoughts.appspot.com/

quickthoughts

There are a lot of other Appengine tutorials including the official tutorial that do a very thorough job of explaining setting up a development environment and deploying.  I’m just going to share some more example code with some helpful comments.

I know looking at code is not pretty, but I’ve included a lot of comments for each part.  It should be easiest to understand in that format.  Ugh: and I’m sorry that the crappy WordPress syntax highlighting has totally f’d the format.  That pretty much makes python code useless, but hopefully you’ll be able to sift through.

Template

There are only three files for this app.  The first is the HTML template for the single page that is used.

index.html

</pre>
<html>

<head>
<title>Quick Thoughts</title>
<!--- The following CSS could ofcourse be put in a seperate file, but this is simplest for now --->
<style>
form { width: 320px; }

body {
font-family: "Trebuchet MS, Verdana, Arial, Helvetica, sans-serif"
font-size: 16px;
}
.date {  font-style: italic; font-size: 12px;  }
.thought { padding-top: 30px; }
div#outer {
 width: 500;
 background-color:#FFFFFF;
 margin-top: 50px;
 margin-bottom: 50px;
 margin-left: auto;
 margin-right: auto;
 padding: 10px;
 //border: thin solid #000000;
}
h2 { margin-bottom: 0px; }
.username {
 margin-bottom: 20px;
}
</style>
</head>

<body>
<div id="outer">
<h2>Quick Thoughts</h2>
<!--- Showing the user's Nickname here.  User object also has .email, and .user_id
 more info here: http://code.google.com/appengine/docs/python/users/userclass.html
 --->
<div>By {{ user.nickname }}</div>
<div>
 <form action="." method="POST">
 <textarea name="thought" rows="6" cols="40"></textarea>
<div align="right"><input type="submit" value="Record" align="right"></input></div>
</form></div>
<!---
 This for loop prints out the different Thoughts stored in the database
 Appengine uses Django .96's templating system which I personally think is pretty great.
 For more information visit http://www.djangoproject.com/documentation/0.96/templates/
 --->
 {% for thought in thoughts %}
<div>
<div>{{ thought.thought|linebreaksbr }}</div>
<div>{{ thought.date|date:"D. N jS g:i a" }}</div>
</div>
{% endfor %}</div>
</body>
</html>
<pre>

This is all pretty straightforward especially if you come from the Django world.  Appengine wisely uses Django’s templating system to render its HTML pages. For more details on the templates view the Django .96 Template Documentation.

Request Handling

The second file is the python WSGI handler.  You can ofcourse use Django on Appengine and have the advantage of the nice url parser and the Django views format but here I just stuck with the WSGI RequestHandlers.

thoughts.py


from google.appengine.api.urlfetch import fetch as urlfetch, GET, POST
from google.appengine.ext import db
from google.appengine.ext.webapp import RequestHandler, WSGIApplication
from google.appengine.ext.webapp import template
from google.appengine.api import users

import os

from wsgiref.handlers import CGIHandler

class Thought(db.Model):
 """
 This is the Database Model that stores the different Thought objects that the user submits
 Each entry in the database stores a thought, date and the user who wrote it.
 """
 thought = db.TextProperty()
 date = db.DateTimeProperty(auto_now_add=True)
 # The auto_now_add setting automatically adds the date that the object was created so you don't have to.
 user = db.UserProperty()  # Google handles the user for you.  Great!

class ThoughtHandler(RequestHandler):

 def get(self):
 user = users.get_current_user() # Get the user
 if not user:
 # If they are not logged in, ask google to authenticate them.
 self.redirect(users.create_login_url(self.request.uri))

 # These are the variables that will be sent to the template
 template_values = {
 # This is a GQL query for the appengine datastore.  
 # Here we're finding all Thoughts for the given User and ordering them by Date descending
 # More info on GQL: http://code.google.com/appengine/docs/python/datastore/gqlreference.html
 'thoughts':  Thought.all().filter("user =", user).order('-date'),  
 'user': user,
 }

 # Gather the full path to the template
 path = os.path.join(os.path.dirname(__file__), 'index.html')

 # Render the template with the template_values we collected above
 html = template.render(path, template_values)

 # Write out the result
 self.response.out.write(html)

 def post(self):
 """
 A Thought has been submitted via POST.
 Create a new Thought object and re-direct back to the front page.
 """
 user = users.get_current_user() # Get the user
 if not user:
 # If they are not logged in, ask google to authenticate them.
 self.redirect(users.create_login_url(self.request.uri))

 # Get the 'thought' POST data from the request
 thought = self.request.get('thought')

 # Create a new thought object using the POST data and the authed user
 t = Thought( thought = thought, user = user )
 # Save the object
 t.put()

 # Now re-direct back to the front page
 self.redirect('/')

def main():
 """
 This simple function is the URL parser
 There's only one URL for this app, so its a pretty bad example for this ;)
 """
 application = WSGIApplication([

 ('/', ThoughtHandler),

 ], debug=True)

 CGIHandler().run(application)

if __name__ == '__main__':
 main()

There are a few major highlights in this code: the Thought Datastore Model, the Query for your Thoughts, and the simple Google Authentication.  I LOVE these three lines of code (yes, I know how nerdy that sounds):

user = users.get_current_user() # Get the user
if not user:
    self.redirect(users.create_login_url(self.request.uri))</pre>

In those 3 lines we’ve requested the User object and asked Google to authenticate them and send them back if they’re not logged in!  Super simple!  No more login/signup/change password/change username crap to deal with here.  The authentication is done for you.

Toward the top is the Thought model that is a subclass of db.Model.  For those of you who’re familiar with Django this format will look familiar.  The Thought model contains the text of the thought, the date it was recorded and the user who recorded it.

In the ‘get’ Request we query for all of the thoughts of the given user and order them by descending date.  The objects are fetched using GQL, the query interface for the Datastore.  You can handle most queries by playing with the format of the above example, but here is more information on GQL.

Configuration

Finally we need the configuration file for our app.  Its called app.yaml and it tells appengine what App we’ve registered as, and how to handle the URLs.

app.yaml


application: YOURAPPNAME
version: 1
runtime: python
api_version: 1

handlers:

- url: /.*
 script: thoughts.py

Other tutorials explain this file well.  You can expand it to include other scripts and serve static files.

Conclusion

Now you have your own private micro-blog on Google’s datastore!  That means it’s theoretically infinitely scalable without you ever having to worry about a thing.  You can grow to the size of twitter and never blink an eye :) .

Hope it helped some people.  Feel free to use this code in any way you’d like and feel free to leave questions, comments or corrections.

Starting Out With Comet (Orbited) Part 2 – Installation and STOMP

In this part of the tutorial we will install and setup the server side of a Comet installation using the Orbited implementation.  We’ll also be using the MorbitQ STOMP server to handle message passing and we’ll play around with STOMP/Comet setup using Orbited’s STOMP Test demo.  If you have not read the first blog post in this series I advise you do so.  I will be assuming you have an understanding of those terms and concepts of the previous post for this tutorial.

Before we start I’d like to quote my source.  I’ve learned most of what I’m sharing from Michael Carter’s Tutorial and many hours of playing around.

Installation

We will need to install the latest version of Orbited and also some Stomp tools.  The stomp tools aren’t required for this step but will be for anything else you want to do with comet so I’ve included them in the installation steps here as well.  Conveniently Orbited is setup in the Cheeseshop.  You need python2.5+ and if you have not installed the python setup-tools do so now.

Installation simply consists of the following commands.

easy_install twisted
easy_install orbited
easy_install stompservice
easy_install simplejson

To test if it works enter your python shell and test importing the libraries.  The following should load without any errors

$ python
>>> import twisted
>>> import orbited
>>> import stompservice
>>> import simplejson

If you have any troubles there is more info on the Orbited Installation Guide.

Configuration

Orbited is configured with a ‘.cfg’ file. Lets call ours example.cfg.  Make a directory anywhere for your project and paste the following into example.cfg

[global]
session.ping_interval = 300

[listen]

http://:9000

stomp://:61613

[access]
* -> localhost:61613

Lets go through the different parts.

The ping interval is a number of seconds for the backend to wait before it pings the client.  We’ve got it set up for 5 minutes.  Good comet implementations have some sort of pinging system.  This is a necessary step as due to current HTTP protocols the client cannot tell if something has gone wrong on the server end.  It simply waits happily all day for some sort of response for the server.  But with a pinging system setup we can tell the client to refresh its connection if it hasn’t heard from the server in the last 300 seconds, and the server will make sure to ping the client at least every 300 seconds, letting it know that the connection is still alive.

The listen parameters tell the orbited server which ports to listen to and who to proxy requests to.  In our configuration port 9000 will be serving static html files, and port 61613 will be a proxy for our STOMP server.

And finally, the access parameter gives permission to proxy to the stomp server.

Lets Run It

To run enter your project directory and type

orbited – -config example.cfg

It should look something like this:

06/24/09 21:05:24:651 INFO   orbited.start    proxy protocol active
06/24/09 21:05:24:511 INFO   orbited.start    Listening http@9000
06/24/09 21:05:24:525 INFO   orbited.start    Listening stomp@61613

STOMP Test

Orbited comes with a really nice STOMP demo that also serves as a nice tool for debugging your setup later.  We’ll use it to play around with Comet and understand the concepts behind STOMP.

While your orbited server is running visit the following URL.

http://localhost:9000/static/demos/stomp/

cometwindowThere are 3 important tools/rows we’ll be using here: Connect, Subscribe and Send.

First click on “Connect” to connect to the orbited and stomp servers.  We’re using MorbitQ has the stomp client, which doesn’t deal with authentication, so any name and password will work.  Notice that the STOMP test shell will now say.

→ Transport openned
→ Connected as user guest

Second change the “destination” in the “Subscribe” row to be “/channel/1/” and click Subscribe.  You have now created and subscribed to a channel called “/channel/1/”.

Next we’ll send something to that channel using the Send tool.  Again change the destination to “/channel/1/” and type something into the message box replacing “hello”.  In the image above I’ve chosen “comet is working!”.  Now hit Send and notice that your message shows up in your STOMP shell!

Try sending to other destinations and notice that only messages sent to “/channel/1/” will show up in the stomp shell.  We can change that however by subscribing to additional channels.  Try subscribing to “/anotherchannel/” and then send it a message.  Notice that this setup can handle being subscribed to many different channels at once.

The Real Power

All of the things we’ve tried so far could have been fairly easily implemented with simple AJAX.  The real power of comet is that it can push information to the client without having to submit a request.  Also, the real power of STOMP is that it smoothly handles message passing between clients.  Lets demonstrate both of these now by opening up multiple browser windows all pointing to our STOMP Test.

stompmultipleOpen up 3 windows and “Connect” them each to the server.  Now subscribe the second window to “/channel/1/”, the third window to “/channel/2/” and the first window to both.

Once setup, using the first window send a message “Message to Channel 1″ to “/channel/1/”.  You’ll notice that it not only showed up instantly in window 1 (where you submitted), but also in window 2 (where you did nothing)!  The STOMP server has passed the message all clients listening to “/channel/1/” and the Comet server has pushed it to the client without it having to poll for updates!

Now send a message to “/channel/2/” and notice that it shows up in windows 1 and 3, but not 2.

Play around with this setup more to become familiar.  Each window can subscribe to any number of channels, and each can send messages to any channel, whether it is subscribed to it or not.

Whats Next

We’ve now setup and tested a Comet implementation.  You can see its benefits and understand how it works with STOMP.  In the next example we’ll work on the client side of the Comet implementation and write a python STOMP client to handle processing and sending the data on a comet server.

In the mean time you may want to look at the other demo’s that came with Orbited or Michael’s Demo.

Update: Part 3 – The Client is now available.

Starting Out With Comet (Orbited) Part 1

This is the first article in a series I’m creating to ease developers into using Comet.  The documentation is severely lacking on every comet implementation I’ve come across which I think is Comet’s biggest limitation at the moment. Hopefully this will help those interested in Comet to struggle less with their implementations than I did/do.

The series will specifically focus on using the Orbited implementation of Comet, but many of the concepts will apply to other Comet implementations as well.

This first post will explain some terms and concepts you need to become familiar with.

Comet

Comet is a term referring to a set of techniques (hacks) that enable the server to push data to the client whenever it wants.  Traditionally the client has to initiate all requests.  If the server wants to send something to the client it has to wait until the client chooses to connect again.

Comet is a set of methods for the server to ping the client.

As a definition, Comet is as arbitrary as the word AJAX.  Try not to get hung up on the word any more than you’d get hung up on defining what AJAX is.

What and Why Orbited

Orbited is a Python based implementation of Comet using the Twisted framework.  There’s not a lot out there discussing the pro’s and cons of the different comet services so its hard to tell which implementation is better than another.  There is clearly not yet a dominant implementation.

I basically chose Orbited because it was the first one I could get up and running thanks to this tutorial, because I love python, and because my implementation will be working with Django.

Orbited is built on the Twisted framework and is event based, so in theory it should scale just fine.

Its important to note that Orbited has both a server side component and a JavaScript library for dealing with the client side of Comet connections as well.

Long-Polling

Long-polling is the technique Orbited uses to maintain a consistent real-time connection between the server and the client, enabling the server to push data to the client whenever it wants. Put simply the client makes a request that is kept open (no immediate response is returned).  The server can continue to send data to the client through this open connection until it is terminated by the client.

Streaming

Streaming is another Comet technique.  Michael Carter, one of Orbited’s main contributors has a nice article on the advantages of the two techniques.

STOMP

Short for Streaming Text Orientated Messaging Protocol, STOMP is is a message passing protocol commonly used in Comet implementations.  It enables a publish/subscribe (pub/sub) model that comes in handy in many real-time applications.

STOMP isn’t a required part of a Comet implementation, but we’re going to be using it in this series.

MorbidQ

MorbidQ is a STOMP server that comes with Orbited.  Think of it as Apache, but for STOMP instead of web apps.  Its great in that its easy to set up and work with, but its not robust enough for a significant load.  When deploying you should instead use RabbitMQ.

Pub/Sub

In the following posts we will be using a publish/subscribe model thanks to the help of STOMP.  With this model you can create channels and both publish and subscribe to them.  This model works great when multiple clients are listening to the same thing, such as a group chat or a game.

I like to think of it as a telegraphy system.  A whole bunch of people can hook up to the wire, and if they do they hear everything everyone else is broadcasting, and can broadcast their own messages as well.

So subscribing is like hooking up your headphones to the transmission lines and publishing is like sending out a signal with your morse key.  A different channel would correspond to a different wire.

Orbited Server

Comet requires an event based server to deal with the long requests.  Typical servers like Apache for instance were designed to handle a request as quickly as possible.  They allocate memory to returning a response as soon as a request is received, and maintain that memory until a response is returned.

An event-based server such as Orbited can chop up the different components of the request/response process.  It can accept a request and then forget about it while allocating its memory elsewhere until at some later point something needs to be done (a response returned or data transmitted).  For this reason it can handle dramatically more open connections than Apache.

Lets use an example.  Pretend we have a typical Apache hosted site that attracts 10,000 people every hour.  Lets say with that amount of traffic we get on average 50 new page requests every second and it takes about a second to return each page.  That means Apache has at any given time about 50 open connections.  That’s what it is designed to do.

But if we now put Comet on that site, all 10,000 of those users need to have a long connection open with our server.  Similar to the first estimate lets say only 50 of them are really receiving any data at any point.  Our Comet server will only assign memory to the tasks of those 50 connections, where Apache would attempt to assign an equal amount of memory to all 10,000 open connections.

That’s why its important in Comet to separate connections from events, or why we use Orbited (or equivalent comet server) instead of Apache.

Continuing…

That’s it for my explanations/intro post.  I’m fairly new to the Comet scene myself and am very busy with my startup.  I’m sure there are a lot of additions or corrections, please leave them in the comments and I will try to keep up with corrections..

I’ll put more time in this depending on how much interest there is in the comments and subscribriptions.  Make sure to check the Comet Category listings for the other posts.

The next post will be an example implementation.

Update:  Continue to the next post in the series: Starting Out With Comet (Orbited) Part 2 – Installation and STOMP

New Blog Design By Amanda Scharlemann

A few weeks ago I published a request for a generous designer to create a unique look for this blog.  Amanda Scharlemann was the first to respond and was a pleasure to work with!

Amanda is a design student at Bethany Lutheran College and shows great potential.  She had never worked with customizing WordPress Templates and the annoying restrictions that come with that, but was eager and quick to learn.  We went through several iterations to narrow down exactly what I was looking for and she spent a lot of time examining the example sites I gave her to get a feel for what I would like.  At each iteration she had significant improvements, helpful advice, and most importantly was open and attentive to feedback.

I dislike design processes where the designer is stubbornly biased in certain directions and completes 90% of the work before you’re allowed to review.  Amanda was not that way at all.  I’m not sure if they teach client-design relationships at Bethany or not, but she was a pleasure to work with.

Check out her resume.

The custom design was launched last week, and so far the results have been great.  I’ve seen a doubling in traffic, and several more comments than usual.  Perhaps  the most impressive statistic so far is that a friend I hadn’t talked to in a while came across my blog while goggling an issue he was having.  My article apparently helped him and he was excited to notice (thanks to the personalized sidebar) that the blog belonged to me!

Over time I will probably tweak things here and there as is my nature but I’m over all very pleased with the new look.

Thanks again Amanda!

Find RSS Feed Links With jQuery

This took me a little while to figure out so I thought I’d share. You can use a jQuery selector to find any RSS links on a page very easily.

The following line will return a list of the RSS link elements.

var link_elements = $('link[type="application/rss+xml"]');

The following snippet will create an array of all the urls to the RSS feeds on the page.

var links = [];
$('link[type="application/rss+xml"]').each(function() { links[links.length] = this.href; });

Google AJAX Feed API Example with Socialbrowse

Today I learned that Google has a really awesome AJAX API for RSS and Atom feeds. It allows you to access RSS or Atom feeds using JavaScript in both JSON and XML format without having to setup your own proxy or deal with anything on the server side.

They have a lot of great examples, but I thought I’d share mine with the usecase of showing the feed of my Socialbrowse shares. In your Socialbrowse settings you can specify a public share name. Once added 3 feeds will be created for you

profile: http://socialbrowse.com/shares/PUBLIC_NAME/
shares rss: http://socialbrowse.com/rss/uname/PUBLIC_NAME/
feed rss: http://socialbrowse.com/rss/socialfeed/PUBLIC_NAME/

where PUBLIC_NAME is the share name you chose.

For this example we’re going to fetch and display the “shares rss” feed using the Google AJAX Feeds API. The example is simulating the Socialbrowse blog widget Zack created a few months ago.

To use the Feeds API you need to get a key.

Now we have everything we need. Create an HTML file and paste in the following code. Make sure to change YOUR_API_KEY, and the feed_title and feed_url variables to your values.

<html>
  <head>
	
	<script>
	  var feed_title = "Dave's Socialbrowse Feed";
	  var feed_url = "http://socialbrowse.com/rss/uname/PUBLIC_NAME/";
	</script>

    <script  type="text/javascript" src="http://www.google.com/jsapi?key=ABQIAAAAmcnSI-mFmfJW8bidL13qfRQHElLAWyCZ_TZ_pzrAvWp3ncTV5hRgCPRM76Ub8GIqowNBQZMVWYastg"></script>
    <script type="text/javascript">

      google.load("feeds", "1");

      function initialize() {
        var feedControl = new google.feeds.FeedControl();
        feedControl.addFeed(feed_url, feed_title);
        feedControl.draw(document.getElementById("feedControl"));
      }
      google.setOnLoadCallback(initialize);

    </script>
  </head>

  <body>
    <div id="feedControl">Loading</div>
  </body>
</html>

Viewing the page in a browser will result in something that looks like this

feed

Its not very pretty but its easy enough to style it however you like using CSS. This code can be added anywhere you’d like to display your Socialbrowse shares.

Finally, note that this code can be used with any feed, not just Socialbrowse. If you want to display the latest Google News for example simply change the feed variables to

	  var feed_title = "Google News";
	  var feed_url = "http://news.google.com/news?pz=1&ned=us&hl=en&topic=h&num=3&output=rss";

TicTacToe in jQuery

As a demo application for a project of mine I wrote TicTacToe in Javascript using the jQuery framework.

I’ve added excessive comments to the code to provide an easy walk-through example on the jQuery/Javascript game. I’m in no way a JavaScript expert, there are a hundred different ways to program TicTacToe, and this code is far from clean but here it is!

There are just two files, the HTML page, and a page holding the javascript.

tictactoe.html


<html>

<head>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.js"></script>
<script src="tictactoe.js"></script>

<style>

table.board {
  border: 1px solid green; 
  height: 600px;
  width: 600px;
}

body {
  text-align: center;
  align: center;
}

td {
  height: 200px;
  width: 200px;
  text-align: center;
  vertical-align: middle;
  font-size: 100px;
  font-weight: bold;
  font-color: green;
  font-family: geniva, verdana, helvetica;
  border: 1px solid green;
}

#menu {
  text-align: center;
  position: absolute;
  width: 400;
  height: 400;
  margin-left: 100px;
  margin-top: 100px;
  border: 5px double red;
  display: none;
  vertical-align: middle;
  background-color: white;
}

#play_again {
  font-size: 20px;
  color: green;
}

</style>


</head>
<body>

<table border="0px" align="center">
<tr><td>
<div id="menu"></div>
<div id="board"></div>
</td></tr>
</table>

</body>


tictactoe.js

/* Main Game Handling class */
var TicTacToe = {
    turn: "O",  // Keeps a record of who's turn it is
    board: ["", "", "", "", "", "", "", "", "", ""],  // Keeps a record of the TicTacToe Board
    win: false, // records who won if the game is over
    
    /* Clears and starts a new game with a new board */
    restartGame: function() {
      // Draw the board
      var board_table = '<table class="board" border="0px" cellpadding="0px" cellspacing="0px" align="center"><tr><td id="ttt0">&nbsp;</td><td id="ttt1">&nbsp;</td><td id="ttt2">&nbsp;</td></tr><tr><td id="ttt3">&nbsp;</td><td id="ttt4">&nbsp;</td><td id="ttt5">&nbsp;</td></tr><tr><td id="ttt6">&nbsp;</td><td id="ttt7">&nbsp;</td><td id="ttt8">&nbsp;</td></tr></table>';
      $("#board").html(board_table);
      $("#menu").hide();
      
      // clear the board
      this.board = ["", "", "", "", "", "", "", "", "", ""];
      
      // Add on-click events to each of the boxes of the board
      $("#board td").click(function(e) {
          TicTacToe.move( e.target.id );
         });

    },

    /* Handles clicks spaces on the board */
    move: function(id) {
      var space = $("#" + id);  // Board space table element
      var num = id.replace("ttt", ""); // # representing the space on the board
    
      // If no one's gone there, and the game isn't over, go there!
      if (!this.board[num] && !this.win) {
        space.html( this.turn );
        this.board[num] = this.turn;
        this.nextTurn();  // End turn
      } 
    },

    /* Iterate turn and check if anyone one yet */
    nextTurn: function() {
      this.turn = (this.turn == "O") ? "X" : "O";
      this.win = this.check4Win();
      if (this.win) {
          this.endGame();
      }
    },

    /* Display who won and options for new games */
    endGame: function() {
    
      if (this.win == "Cat") {
          $("#menu").html("Cats Game.");
      } else {
          $("#menu").html(this.win + " wins!");
      }
      $("#menu").append("<div id='play_again'>Play Again</div>");
      
      // Button for playing again.
      $("#play_again").click(function () { TicTacToe.restartGame();  });
      $("#menu").show();
      this.win = false;
    
    },

    // If any of these patters of board spaces have all X's or all O's somebody won!
    wins: [[0,1,2], [3,4,5], [6,7,8], [0,3,6], [1,4,7], [2,5,8], [0,4,8], [6,4,2]],
    
    /* Check for whether someone won the game of it was a Cat's game. */
    check4Win: function() {
        
      // Loop through all possible winning combinations 
      for (k in this.wins){
        var pattern = this.wins[k];
        var p = this.board[pattern[0]] + this.board[pattern[1]] + this.board[pattern[2]];
        if (p == "XXX") {
          return "X";  // X Won!
        } else if (p == "OOO") {
          return "O";  // O Won!
        }
      }
      
      // Check if all spaces in the board are filled, then its a Cat's game
      var cnt = 0;
      for (s in this.board) {
        if (this.board[s]) { cnt+=1; }
      }
      if (cnt == 9) { 
        return "Cat";  // Cat's game!
      }
  }
};

$(document).ready(function() {
    
    // Start a game!
    TicTacToe.restartGame();
});

I’d originally setup the game on AppJet but unfortunately today they announced that they are closing down their framework and free hosting for a while to focus on one of their successful apps EtherPad. For at least the next month however you can play the game here, and play with the source code here. Feel free to use my code in any way.

If you do use the source for something, or have suggestions on improvements make sure to leave a comment.

GNU Screen Basic Tutorial

If you’ve ever SSHed into a server, ran something, logged off and got frustrated because whatever you were running stopped running because you had to change coffee shops or go to class then you’ve experienced the need for GNU Screen.

Put simply it allows you to maintain a persistent session, a terminal that does not change or quit just because you logged out.  Installation is easy.  On a debian machine its simply

sudo apt-get install screen

Usage is straight forward as well.  Type the word “screen”

$ screen

and you’ll be shown a welcome window, press space to get through.  Now you’ll simply see a standard shell.  You can execute whatever you want here and the session will not die until you tell it to.  You can start many different sessions and toggle between them.  Here’s a table of the basic controls for screen.

  • Ctl-a Ctl-c, create a new window
  • Ctl-a Ctl-a,  switch between windows
  • Ctl-a n, toggle to next window
  • Ctl-a p, toggle to previous window
  • Ctl-a 5, toggle to 5th window
  • Ctl-a “, get a menu listing all of the window
  • Ctl-a A, tool for adding a name to your window, helpful for the menu view
  • Ctl-a k, Kills the window
  • Ctl-a d, Detach from the session

Another thing to note.  Once you detach from a screen, you need to re-attache the next time you run screen.  To do that simply specify the -r parameter.

$ screen -r

Update 6-3-2009: A few fixes were made thanks to Ben Finney’s comments

Deploying MoinMoin on Ubuntu using Apache mod_wsgi

I just went through a somewhat lengthy setup process to deploy a moinmoin wiki on Ubuntu. There’s a lot of documentation on it which actually makes it take a bit longer than I’m used to for getting something running on Ubuntu.  I thought I’d share my streamlined notes for this common deployment scenario.

The first step is configuration.  Greater detail on all of this can be found here.

wget http://static.moinmo.in/files/moin-1.8.3.tar.gz
tar -xzvf moin-1.8.3.tar.gz
cd moin-1.8.3

python setup.py install --prefix='/usr/local' --record=install.log

#Setup the variables.
export PREFIX=/usr/local
export SHARE=$PREFIX/share/moin
export WIKILOCATION=/path/to/wikis
export INSTANCE=your_wiki_name
export GROUP=www-data
export USER=www-data

# Now it copies the default data
cd $WIKILOCATION
mkdir -P $INSTANCE                   # make a directory for this instance
cp -R $SHARE/data $INSTANCE       # copy template data directory
cp -R $SHARE/underlay $INSTANCE   # copy underlay data directory
cp $SHARE/config/wikiconfig.py $INSTANCE   # copy wiki configuration sample file

# Set the permissions
chown -R $USER.$GROUP $INSTANCE   # check that USER and GROUP are correct
chmod -R ug+rwX $INSTANCE         # USER.GROUP may read and write
chmod -R o-rwx $INSTANCE          # everybody else is rejected

# Copy over the server config files
cp/usr/local/share/moin/server/*.wsgi $INSTANCE
cp/usr/local/share/moin/server/*.cgi $INSTANCE

# If you want everyone (not just admins) to be able to edit it you need to run this command
# chmod -R a+rwX $INSTANCE

Now, if you haven’t already install apache and mod_wsgi

sudo apt-get install apache2 libapache2-mod-wsgi

Next edit your apache httpd.conf file ( /etc/apache2/httpd.conf ) and add the following to the end. More help on this step can be found here.

LoadModule wsgi_module modules/mod_wsgi.so  # Loads mod_wsgi
<VirtualHost *>
ServerAdmin youremail@example.com
ServerName wiki.example.com
Alias /moin_static183/ "/usr/local/share/moin/htdocs/"
#ScriptAlias /developers "/path/to/wikis/your_wiki_name/moin.cgi"
WSGIScriptAlias    / /path/to/wikis/your_wiki_name/moin.wsgi
WSGIDaemonProcess developerwiki user=www-data group=www-data home=/root process=5 threads=10 maximum-requests=1000 umask=0007
WSGIProcessGroup developerwiki
</VirtualHost>

And finally you need to add your wiki directory to the python path.

echo "/path/to/wikis/your_wiki_name/" > /usr/lib/python2.5/site-packages/wikis.pth

Restart Apache and you should be set up.

Follow

Get every new post delivered to your Inbox.