I use a lot of doctests for apps that all need to work on a set of initialized data.  I was hoping that there would be some kind of hook in Django for this but there is not.

I could switch all of the doctests to unittests and use fixtures but that would be a lot of work and I prefer doctests.  I could also go through and paste some sort of init command at the beginning of each test that would ensure the data was loaded or do the loading but that’s just plain bad practice.

I came up with a method of creating a ‘testsetup’ app that is always run before the other apps ensuring that whatever that app configures or loads into the database will be run first before any other apps preform their tests.  Here’s how you can do it too.

First create a ‘testsetup’ app and edit its tests.py file

./manage.py startapp testsetup
open testsetup/tests.py

__test__ = {"initialize tests": """

>>> your init code here

""" }

In the test.py file you can load the database, prime the cache, or setup whatever else you need initialized.  Then add the ‘testapp’ as the first app to your settings.py


INSTALLED_APPS = (

'testsetup',

...

)

Now whenever you run

./manage.py test

It will first run the tests for the ‘testssetup’ script and everything will be primed.  If that’s the only kind of test you run then that’s all you’ll need.  You’re done with this tutorial.

But if you run app level tests (ie. ./manage.py test someapp anotherapp ) then the above solution is not enough.  To ensure the testsuite is run before these apps we’ll make our own TEST_RUNNER.  Create a file called ‘testrunner.py’ with the following source.


def run_tests(test_labels, verbosity=1, interactive = True, extra_tests=[]):
print "Given these test_labels", test_labels
print "With these extra_test", extra_tests

from django.test.simple import run_tests as django_run_tests
if test_labels:
# Make sure 'testsetup' is run first
tl = ['testsetup']
tl.extend(list(test_labels))
test_labels = tuple(tl)
print "Testing these apps:", test_labels

django_run_tests(test_labels, verbosity, interactive, extra_tests)

and then in your settings.py file set the TEST_RUNNER variable


TEST_RUNNER = 'testrunner.run_tests'

This script simply wraps the django test runner to ensure that the testsetup app is tested before any other apps are.  It basically makes the django test runner think that you’re running ./manage.py test testsetup someapp when you actually run ./manage.py test someapp.

Proposal

A note to the community: I think it’d be great if the Django settings.py included a TEST_INIT variable which allowed you to point to a function that would be executed immediately before the first test was run.  The hook would make the setup process for doctests much easier.

Google Appengine currently does not allow configuration of naked domains.  Meaning, if you’re building something on Appengine you’ll have to settle for a URL like http://www.mysite.com or http://whatever.mysite.com and you will not be able to use http://mysite.com.

Its not so bad in most cases, but not having http://mysite.com isn’t going to stop people from trying to go there.  So its important to set up some sort of device to forward the naked domain to the www.

Unfortunately you’re going to need a server and for this example it will need to be running Apache2.  The redirect is handled by placing the following line in an apache config file (/apache2/httpd.conf).

Redirect permanent / http://www.mysite.com/

Where http://www.mysite.com is the example site being hosted on Appengine.

Or, if your apache server is hosting other apps and domains you’ll need to set up the redirect in a VirtualHost as shown here.

<VirtualHost *:80>
ServerName mysite.com
 Redirect permanent / http://www.mysite.com/
</VirtualHost>

Note that the ‘permanent’ parameter in the Redirect command enforces a 301 or permanent redirect.  You can choose other forms of redirect by replacing the ‘permanent’ with either the redirect number (ie 302) or the keywords from the following table I’ve shamelessly copied from here.

HTTP Code Status Description
301 permanent The resource has permanently moved
302 temp The resource has temporarily moved
303 seeother The resource has been replaced and refer to new resource
305 UseProxy Use proxy to access site
307 Temp The resource has temporarily moved
410 Tegone The resource has permanently removed

Finally set your DNS to point the base domain to this apache server and in a few hours it should be permanently re-directing http://mysite.com/ to http://www.mysite.com/.

If you’re not using apache or are looking for more ideas here’s a list of other techniques.

Also, please vote for Google to fix the issue here and here.

Yesterday I was on the cover of the Business section of the StarTribune.  Its a nice article that talks a bit about Socialbrowse and my path to what I’m doing right now.

It also talks about Luke Francl and his awesome startup FanChatter which is also from Minnesota and also in YCombinator.  Here is a link to the article

http://www.startribune.com/business/52719012.html

The print version comes with pretty pictures of Luke and I which I think is worth the money :).

Thanks Thomas Lee for the nice coverage.

Today I needed to write a wrapper around a program that would examine the stdio and respond immediately to certain results.  The task was handled nicely with a python script using subprocess.

There isn’t a lot of documentation examples on it so to figure it out I wrote this small test app and I thought I’d share it here.

To begin I wrote a simple script that prints out “Program has run for x seconds.” every x seconds to simulate a long running and noisy script that I might be observing.  Here is the code.

sample.py

import sys, time

class FlushFile(object):
   """Write-only flushing wrapper for file-type objects."""
   def __init__(self, f):
       self.f = f
   def write(self, x):
       self.f.write(x)
       self.f.flush()

# Replace stdout with an automatically flushing version
sys.stdout = FlushFile(sys.__stdout__)

for i in xrange(100):
   sys.stdout.write("Program has run for %d seconds.\n" % i)
   time.sleep(1)

The main part of the program is the last 3 lines where we write “Program has run for %d seconds” 100 times and pause for a second between each.

The FlushFile object is simply a nice hack to overwrite the default stdout object to ensure that the buffer is flushed every time it is written to.  Without the hack our log checker would simply be hung up until the task is fully completed instead of being able to read each line of the output as it is written.

The output looks like this

$ python sample.py
Program has run for 0 seconds.
Program has run for 1 seconds.
Program has run for 2 seconds.
.... # and on until 100 seconds

Next I wrote the wrapper which executes the above sample.py script as a python subprocess and watches the output.  For this example I check whether a ‘4’ exists in the output.  If it is I restart the script therefore creating a continual loop counting between 0 and 4 seconds.

observer.py


import subprocess, os, signal
cmnd = "python sample.py" #change this line to run your script
p, line = True, 'start'

while True:
   p = subprocess.Popen(cmnd, shell=True, stdout=subprocess.PIPE)

   while line:
       line = p.stdout.readline()
       print "line is:", line
       if line.count('4'):
           print "restarting the process"
           os.kill(p.pid, signal.SIGUSR1)
           line = True
           break

   del p

The output is continually checked in the ‘while line:’ loop of line 8.  In this example its printed out for our convenience.  Like 11 checks for the character ‘4’ somewhere in the output.  If it does exist it kills the process and the stdout reading loop.

That loop is wrapped in another however that simply restarts the process all over again.  The resulting output is

$ python observer.py
line is: Program has run for 0 seconds.
line is: Program has run for 1 seconds.
line is: Program has run for 2 seconds.
line is: Program has run for 3 seconds.
line is: Program has run for 4 seconds.
restarting the process
line is: Program has run for 0 seconds.
line is: Program has run for 1 seconds.
line is: Program has run for 2 seconds.
line is: Program has run for 3 seconds.
line is: Program has run for 4 seconds.
restarting the process
line is: Program has run for 0 seconds.
.....

So now we have a nice template for wrapping and responding to executables!

Comet implementations require both server and client side components.  In Part 2 of this series we installed and configured the server side component and then used the example STOMP Test client to test it out and get a feel for what was going on.

In this part of the series we’ll be covering the client side to make our custom interface to the comet server.  To demonstrate we’ll go through the steps in creating an app I call EZChat which is basically a bare bones comet chat client.  The interface will allow you to choose a name and submit messages that will be broadcast to everyone viewing the page in real time.

For it to work you will have to have the configuration of Part 2 setup and your Orbited server running.  For reference I learned most of the client side code in this example by playing with the source of the example STOMP Test client. I recommend taking a look at that source if you need more advanced options or another example.

Includes

At the top of the <head> you first need to include the libraries for the Orbited (/static/Orbited.js) and STOMP (/static/protocols/stomp/stomp.js) client-side implementations.


<script>document.domain=document.domain</script>
 <script src="http://localhost:9000/static/Orbited.js"></script>
 <script>
 Orbited.settings.port = 9000;
 TCPSocket = Orbited.TCPSocket;
 </script>
 <script src="http://localhost:9000/static/protocols/stomp/stomp.js"></script>

 <script src="http://www.json.org/json2.js"></script>

In between the scripts we’ve set up a TCPScocket.  It must be don between the two script includes because the STOMP library needs the socket setup for its execution.  We’ve also specified the Orbited port which is necessary especially if you change the port on which the orbited and stomp javascript files are hosted on (they can both be hosted on port 80 along with your other scripts).

Lastly, we’ve also included a popular JSON library.  Typically I use jquery-json but I’ve kept this tutorial free of javascript frameworks so as not to add unneeded complexity.  If you’d like you can of course switch to whatever JSON library you’re used to; just replace the JSON.stringify and JSON.parse functions with your equivalents.

STOMP Setup

Because we’re dealing with the setup we’ll skip to the bottom of the page and add the following script just before the end tag of the body (</body>).


<script type="text/javascript">
(function() { // set up stomp client.

stomp = new STOMPClient();

 stomp.onconnectedframe = function() {  // Run on initial connection to STOMP (comet) server
 stomp.ready = true;
 // subscribe to channel CHANNEL = "/ezchat/"
 var CHANNEL = '/ezchat/'
 stomp.subscribe(CHANNEL);
 };

 stomp.onmessageframe = function(frame) {  // Executed when a messge is received
 my_receive( JSON.parse(frame.body) );
 };

 // Everything is setup. Start the connection!
stomp.connect(document.domain, 61613); //, 'guest', 'guest');
})();
</script>

My apologies as always for the crappiness of the wordpress syntax parser.   Lets walk through what’s happening.

At the top, we initialize a new STOMPClient object.  The StompClient has the following hooks you can override to trigger your own events.

onopen – Called when the Transport is opened
onclose – Called when the Transport has closed
onerror – Called when the Stomp Client has errored
onerrorframe – Called when there is an error in the message received
onconnectedframe – Called when a the client is fully set up for sending/receiving
onmessageframe – Called when a message is received

The STOMP object also has these functions for connecting and resetting the connections.

reset – Resets the STOMP connection
connect – Connects to the STOMP server
send – Sends the object in the first argument to the channel specified by the second argument

In our simple example only the onconnectedframe and onmessage frames need to be overwritten.

The onconnectedframe function is called when the STOMP server has been connected to and everything is setup for sending and receiving messages.  Inside this function we simply need to subscribe/listen to the CHANNEL we’ve setup for our chat.  For the example I’ve chosen the channel ‘/ezchat/’.  Once subscribed our STOMP client will receive any messages sent to that channel in real time.  You can subscribe to multiple channels if you’d like, and you can make clients with different channels if you’d like to have different chat rooms.  But for this example we’ll just stick with the hard coded ‘/ezchat/’ channel.

The onmesageframe function is called when a message has been received.  It is passed a frame object with the following structure

frame

  • body: “{“name”:”Dave”,”message”:”awesome this is working”}”
  • headers: Object
    • content-encoding: “utf-8”
    • content-length: “51”
    • content-type: “text/plain”
    • destination: “/ezchat/”
    • message-id: “/ezchat/_3”
  • type: “MESSAGE”

where the body holds the information that has been sent.  The STOMP server and client add extra “type” and “headers” objects to communicate between each other.  The extra information can be very useful for more complicated applications but for our simple example we’re only interested in the frame “body”.

So you see that the onmessageframe is simply parsing the json object in frame.body of every received message and passing it to my_receive, a function we will soon create.

The Content

For EZChat we need a form where users can specify a name and type messages to send.  We also need an area to put the messages.  Plop this HTML in at the top of the <body> to handle all of that.

<h2>EZChat - Example Comet Client!</h2>
Everyone viewing this page will see the messsages you submit instantly.
<form id="message_form" action="#">
 Name:
 <input type="text" name="chat_name" id="chat_name"></input>
 Message:
 <textarea name="message" id="message" rows="4" cols="40"></textarea>
 <input type="submit" name="Send" onclick="return my_send(); return false"></input>
</form>
<div id="messages"><!--- All received messages will get placed here ---></div>

There are a few things to notice here.  First, the important elements in the form have ids ‘chat_name’ and ‘message’ and the area that will be containing all the received messages is called ‘messages’.  The names don’t matter except that we’ll use them in the functions we create later.

Second, the onclick event of the submit button is overridden with instead calling the my_send function.  We’ll make this function in the next step.

The Functions

Lastly we need to make the custom my_send and my_receive functions that get called to send and receive messages.  Insert these functions into the head after the includes.

my_send

The my_send function will get the values from the ‘chat_name’ and ‘message’ form elements, combine them in an object, convert the object to json, and then sends it to the ‘/ezchat/’ channel.  The sending is handled using the stomp.send command which takes as input the object to send and second, the channel to send it to.

var CHANNEL = '/ezchat/';
function my_send() { 

// Get the values to send from the form
 var name = document.getElementById('chat_name').value;
 var message = document.getElementById('message').value;

 var msg = {'name': name, 'message': message};

 var json_msg = JSON.stringify(msg);
 stomp.send(json_msg, CHANNEL)
 return false;
 }

my_receive

As discussed earlier the my_receive function gets the JSON parsed version of whatever was sent in ‘frame.body’. In the case of our app its always an object of the format

msg = {‘name’: <some name>, ‘message’: <some message>}

The my_receive function simply takes this object and converts it into a prettier HTML format and appends it to the top of the message list we created in the HTML.

function my_receive( msg ) {
 console.log('received message', msg);
 // append the <msg> to the top of the list of messages.
 var messages_el = document.getElementById('messages');
 var new_message = "
<div><strong>" + msg['name'] + ":</strong> " + msg['message'] + "</div>
";
 messages_el.innerHTML = new_message + messages_el.innerHTML;
 }

That’s it for the code.  Scroll to the bottom of the page for the full version of the source.

Results

Ensure that your Orbited server is running as described in Part 2 of the series.  Then load up the page we’ve made in two or more separate windows.  Choose a different name for each window and start sending each other messages.  You’ll notice that both windows will receive the submitted messages almost instantaneously!  The beauty of Comet!

ezchatIf you open your STOMP Test Client and subscribe to the ‘/ezchat/’ channel you’ll see a more raw input on what’s actually being received by the STOMP clients as you chat.

ezchatstompYou can see how the STOMP Test client is incredibly handy for debugging.

That’s it for this part of the Tutorial!  There are still parts to come including writing a data handler on the server side and hopefully a much requested post on Django integration.

Full Source

For your convenience here’s the full index.html file for this example.


 <html>

<head>
 <script>document.domain=document.domain</script>
 <script src="http://localhost:9000/static/Orbited.js"></script>
 <script>
 Orbited.settings.port = 9000;
 TCPSocket = Orbited.TCPSocket;
 </script>
 <script src="http://localhost:9000/static/protocols/stomp/stomp.js"></script>

 <script src="http://www.json.org/json2.js"></script>

 <script type="text/javascript">
 // These are our custom functions for sending and receiving STOMP messages.
 // They will be sent in the format msg = {'name': somename, 'message': somemessage}

 var CHANNEL = '/ezchat/';

 function my_receive( msg ) {
 console.log('received message', msg);
 // append the <msg> to the top of the list of messages.
 var messages_el = document.getElementById('messages');
 var new_message = "
<div><strong>" + msg['name'] + ":</strong> " + msg['message'] + "</div>
";
 messages_el.innerHTML = new_message + messages_el.innerHTML;
 }
 function my_send() {
 // Get the values to send from the form      

 var name = document.getElementById('chat_name').value;
 var message = document.getElementById('message').value;

 var msg = {'name': name, 'message': message};
 console.log(msg);

 var json_msg = JSON.stringify(msg);
 console.log(json_msg);
 stomp.send(json_msg, CHANNEL)
 return false;
 }       
 </script>

</head>

<body>
<h2>EZChat - Example Comet Client!</h2>
<div>Everyone viewing this page will see the messsages you submit instantly.</div>
<form id="message_form" action="#">
 Name:
 <input type="text" name="chat_name" id="chat_name"></input>
 Message:
 <textarea name="message" id="message" rows="4" cols="40"></textarea>
 <input type="submit" name="Send" onclick="return my_send(); return false"></input>
</form>
<div id="messages"><!--- All received messages will get placed here ---></div>
<script type="text/javascript">
(function() { // set up stomp client.
 stomp = new STOMPClient();
 stomp.onconnectedframe = function() {  // Run on initial connection to STOMP (comet) server
 stomp.ready = true;
 // subscribe to channel CHANNEL = "/ezchat/"
 stomp.subscribe(CHANNEL);          
 };

 stomp.onmessageframe = function(frame) {  // Executed when a messge is received
 console.log('frame is', frame);
 my_receive( JSON.parse(frame.body) );
 };

 // Everything is setup. Start the connection!
 stomp.connect(document.domain, 61613); //, 'guest', 'guest');
})();
</script>

</body>

</html>

As usual, if there are corrections or questions please be sure to leave them in the comments. And remember to subscribe to catch the rest of the series.

I’ve made a few apps on Google’s Appengine now and am getting to the point where I can pump them out fairly quickly.  I really love that they make user authentication (my least favorite part of web applications) incredibly simplified.

I wrote this app in less than an hour as a simple tool for myself and to test out the authentication tools which I hadn’t gotten a chance to use yet.  I’m sharing the source here in case its of any use to others.

Description

The app is called Quick Thoughts and its a very simple private micro blog (a private twitter).  Basically you log in and can record quick notes to yourself.  They’re dated and only you can see them.

Its all on one page.  You can get a good idea of what it is from this screen shot.  You can also just try it out yourself by logging into: http://quoughts.appspot.com/

quickthoughts

There are a lot of other Appengine tutorials including the official tutorial that do a very thorough job of explaining setting up a development environment and deploying.  I’m just going to share some more example code with some helpful comments.

I know looking at code is not pretty, but I’ve included a lot of comments for each part.  It should be easiest to understand in that format.  Ugh: and I’m sorry that the crappy WordPress syntax highlighting has totally f’d the format.  That pretty much makes python code useless, but hopefully you’ll be able to sift through.

Template

There are only three files for this app.  The first is the HTML template for the single page that is used.

index.html

</pre>
<html>

<head>
<title>Quick Thoughts</title>
<!--- The following CSS could ofcourse be put in a seperate file, but this is simplest for now --->
<style>
form { width: 320px; }

body {
font-family: "Trebuchet MS, Verdana, Arial, Helvetica, sans-serif"
font-size: 16px;
}
.date {  font-style: italic; font-size: 12px;  }
.thought { padding-top: 30px; }
div#outer {
 width: 500;
 background-color:#FFFFFF;
 margin-top: 50px;
 margin-bottom: 50px;
 margin-left: auto;
 margin-right: auto;
 padding: 10px;
 //border: thin solid #000000;
}
h2 { margin-bottom: 0px; }
.username {
 margin-bottom: 20px;
}
</style>
</head>

<body>
<div id="outer">
<h2>Quick Thoughts</h2>
<!--- Showing the user's Nickname here.  User object also has .email, and .user_id
 more info here: http://code.google.com/appengine/docs/python/users/userclass.html
 --->
<div>By {{ user.nickname }}</div>
<div>
 <form action="." method="POST">
 <textarea name="thought" rows="6" cols="40"></textarea>
<div align="right"><input type="submit" value="Record" align="right"></input></div>
</form></div>
<!---
 This for loop prints out the different Thoughts stored in the database
 Appengine uses Django .96's templating system which I personally think is pretty great.
 For more information visit http://www.djangoproject.com/documentation/0.96/templates/
 --->
 {% for thought in thoughts %}
<div>
<div>{{ thought.thought|linebreaksbr }}</div>
<div>{{ thought.date|date:"D. N jS g:i a" }}</div>
</div>
{% endfor %}</div>
</body>
</html>
<pre>

This is all pretty straightforward especially if you come from the Django world.  Appengine wisely uses Django’s templating system to render its HTML pages. For more details on the templates view the Django .96 Template Documentation.

Request Handling

The second file is the python WSGI handler.  You can ofcourse use Django on Appengine and have the advantage of the nice url parser and the Django views format but here I just stuck with the WSGI RequestHandlers.

thoughts.py


from google.appengine.api.urlfetch import fetch as urlfetch, GET, POST
from google.appengine.ext import db
from google.appengine.ext.webapp import RequestHandler, WSGIApplication
from google.appengine.ext.webapp import template
from google.appengine.api import users

import os

from wsgiref.handlers import CGIHandler

class Thought(db.Model):
 """
 This is the Database Model that stores the different Thought objects that the user submits
 Each entry in the database stores a thought, date and the user who wrote it.
 """
 thought = db.TextProperty()
 date = db.DateTimeProperty(auto_now_add=True)
 # The auto_now_add setting automatically adds the date that the object was created so you don't have to.
 user = db.UserProperty()  # Google handles the user for you.  Great!

class ThoughtHandler(RequestHandler):

 def get(self):
 user = users.get_current_user() # Get the user
 if not user:
 # If they are not logged in, ask google to authenticate them.
 self.redirect(users.create_login_url(self.request.uri))

 # These are the variables that will be sent to the template
 template_values = {
 # This is a GQL query for the appengine datastore.  
 # Here we're finding all Thoughts for the given User and ordering them by Date descending
 # More info on GQL: http://code.google.com/appengine/docs/python/datastore/gqlreference.html
 'thoughts':  Thought.all().filter("user =", user).order('-date'),  
 'user': user,
 }

 # Gather the full path to the template
 path = os.path.join(os.path.dirname(__file__), 'index.html')

 # Render the template with the template_values we collected above
 html = template.render(path, template_values)

 # Write out the result
 self.response.out.write(html)

 def post(self):
 """
 A Thought has been submitted via POST.
 Create a new Thought object and re-direct back to the front page.
 """
 user = users.get_current_user() # Get the user
 if not user:
 # If they are not logged in, ask google to authenticate them.
 self.redirect(users.create_login_url(self.request.uri))

 # Get the 'thought' POST data from the request
 thought = self.request.get('thought')

 # Create a new thought object using the POST data and the authed user
 t = Thought( thought = thought, user = user )
 # Save the object
 t.put()

 # Now re-direct back to the front page
 self.redirect('/')

def main():
 """
 This simple function is the URL parser
 There's only one URL for this app, so its a pretty bad example for this 😉
 """
 application = WSGIApplication([

 ('/', ThoughtHandler),

 ], debug=True)

 CGIHandler().run(application)

if __name__ == '__main__':
 main()

There are a few major highlights in this code: the Thought Datastore Model, the Query for your Thoughts, and the simple Google Authentication.  I LOVE these three lines of code (yes, I know how nerdy that sounds):

user = users.get_current_user() # Get the user
if not user:
    self.redirect(users.create_login_url(self.request.uri))</pre>

In those 3 lines we’ve requested the User object and asked Google to authenticate them and send them back if they’re not logged in!  Super simple!  No more login/signup/change password/change username crap to deal with here.  The authentication is done for you.

Toward the top is the Thought model that is a subclass of db.Model.  For those of you who’re familiar with Django this format will look familiar.  The Thought model contains the text of the thought, the date it was recorded and the user who recorded it.

In the ‘get’ Request we query for all of the thoughts of the given user and order them by descending date.  The objects are fetched using GQL, the query interface for the Datastore.  You can handle most queries by playing with the format of the above example, but here is more information on GQL.

Configuration

Finally we need the configuration file for our app.  Its called app.yaml and it tells appengine what App we’ve registered as, and how to handle the URLs.

app.yaml


application: YOURAPPNAME
version: 1
runtime: python
api_version: 1

handlers:

- url: /.*
 script: thoughts.py

Other tutorials explain this file well.  You can expand it to include other scripts and serve static files.

Conclusion

Now you have your own private micro-blog on Google’s datastore!  That means it’s theoretically infinitely scalable without you ever having to worry about a thing.  You can grow to the size of twitter and never blink an eye :).

Hope it helped some people.  Feel free to use this code in any way you’d like and feel free to leave questions, comments or corrections.

In this part of the tutorial we will install and setup the server side of a Comet installation using the Orbited implementation.  We’ll also be using the MorbitQ STOMP server to handle message passing and we’ll play around with STOMP/Comet setup using Orbited’s STOMP Test demo.  If you have not read the first blog post in this series I advise you do so.  I will be assuming you have an understanding of those terms and concepts of the previous post for this tutorial.

Before we start I’d like to quote my source.  I’ve learned most of what I’m sharing from Michael Carter’s Tutorial and many hours of playing around.

Installation

We will need to install the latest version of Orbited and also some Stomp tools.  The stomp tools aren’t required for this step but will be for anything else you want to do with comet so I’ve included them in the installation steps here as well.  Conveniently Orbited is setup in the Cheeseshop.  You need python2.5+ and if you have not installed the python setup-tools do so now.

Installation simply consists of the following commands.

easy_install twisted
easy_install orbited
easy_install stompservice
easy_install simplejson

To test if it works enter your python shell and test importing the libraries.  The following should load without any errors

$ python
>>> import twisted
>>> import orbited
>>> import stompservice
>>> import simplejson

If you have any troubles there is more info on the Orbited Installation Guide.

Configuration

Orbited is configured with a ‘.cfg’ file. Lets call ours example.cfg.  Make a directory anywhere for your project and paste the following into example.cfg

[global]
session.ping_interval = 300

[listen]
http://:9000
stomp://:61613

[access]
* -> localhost:61613

Lets go through the different parts.

The ping interval is a number of seconds for the backend to wait before it pings the client.  We’ve got it set up for 5 minutes.  Good comet implementations have some sort of pinging system.  This is a necessary step as due to current HTTP protocols the client cannot tell if something has gone wrong on the server end.  It simply waits happily all day for some sort of response for the server.  But with a pinging system setup we can tell the client to refresh its connection if it hasn’t heard from the server in the last 300 seconds, and the server will make sure to ping the client at least every 300 seconds, letting it know that the connection is still alive.

The listen parameters tell the orbited server which ports to listen to and who to proxy requests to.  In our configuration port 9000 will be serving static html files, and port 61613 will be a proxy for our STOMP server.

And finally, the access parameter gives permission to proxy to the stomp server.

Lets Run It

To run enter your project directory and type

orbited – -config example.cfg

It should look something like this:

06/24/09 21:05:24:651 INFO   orbited.start    proxy protocol active
06/24/09 21:05:24:511 INFO   orbited.start    Listening http@9000
06/24/09 21:05:24:525 INFO   orbited.start    Listening stomp@61613

STOMP Test

Orbited comes with a really nice STOMP demo that also serves as a nice tool for debugging your setup later.  We’ll use it to play around with Comet and understand the concepts behind STOMP.

While your orbited server is running visit the following URL.

http://localhost:9000/static/demos/stomp/

cometwindowThere are 3 important tools/rows we’ll be using here: Connect, Subscribe and Send.

First click on “Connect” to connect to the orbited and stomp servers.  We’re using MorbitQ has the stomp client, which doesn’t deal with authentication, so any name and password will work.  Notice that the STOMP test shell will now say.

→ Transport openned
→ Connected as user guest

Second change the “destination” in the “Subscribe” row to be “/channel/1/” and click Subscribe.  You have now created and subscribed to a channel called “/channel/1/”.

Next we’ll send something to that channel using the Send tool.  Again change the destination to “/channel/1/” and type something into the message box replacing “hello”.  In the image above I’ve chosen “comet is working!”.  Now hit Send and notice that your message shows up in your STOMP shell!

Try sending to other destinations and notice that only messages sent to “/channel/1/” will show up in the stomp shell.  We can change that however by subscribing to additional channels.  Try subscribing to “/anotherchannel/” and then send it a message.  Notice that this setup can handle being subscribed to many different channels at once.

The Real Power

All of the things we’ve tried so far could have been fairly easily implemented with simple AJAX.  The real power of comet is that it can push information to the client without having to submit a request.  Also, the real power of STOMP is that it smoothly handles message passing between clients.  Lets demonstrate both of these now by opening up multiple browser windows all pointing to our STOMP Test.

stompmultipleOpen up 3 windows and “Connect” them each to the server.  Now subscribe the second window to “/channel/1/”, the third window to “/channel/2/” and the first window to both.

Once setup, using the first window send a message “Message to Channel 1” to “/channel/1/”.  You’ll notice that it not only showed up instantly in window 1 (where you submitted), but also in window 2 (where you did nothing)!  The STOMP server has passed the message all clients listening to “/channel/1/” and the Comet server has pushed it to the client without it having to poll for updates!

Now send a message to “/channel/2/” and notice that it shows up in windows 1 and 3, but not 2.

Play around with this setup more to become familiar.  Each window can subscribe to any number of channels, and each can send messages to any channel, whether it is subscribed to it or not.

Whats Next

We’ve now setup and tested a Comet implementation.  You can see its benefits and understand how it works with STOMP.  In the next example we’ll work on the client side of the Comet implementation and write a python STOMP client to handle processing and sending the data on a comet server.

In the mean time you may want to look at the other demo’s that came with Orbited or Michael’s Demo.

Update: Part 3 – The Client is now available.