Archive for the ‘tornadoweb’ tag
Massaging a Tornado web pain point: restart requirement
I have been playing around with the Tornado web framework and I really like it: the shallow learning curve paired with everything it brings to the table makes for a very good framework to at least kick off your next real time app.
One of the core assumptions in tornado is that request are handled quickly: if there are any blocking calls in your request handlers, it will cause other requests to queue. So, I have http wrappers around my dbs and queues so that I can handle these blocking calls from my request handlers asynchronously.
Well and good you say. What is the problem? The pain point is that nodes need to be restarted in order for code changes to propagate. And though it’s not a huge problem, its started to bug me that I have to waste seconds(!) restarting three nodes every time I make a change. So, I wrote a quick python script to do so. It takes the simplest approach; polling for changes in current directory and restarting nodes if required. I was going to use inotify, but as OSX apparently doesn’t have it (has something called FSEvents), I decided to put off learning a new lib for another day so I could keep hacking on my project.
# Simple script to poll all files of interest below the current working directory # for changes. On change, it will run w/e commands you want. For me, it has # been helpful in restarting tornado web nodes. import os import re import time import signal from subprocess import Popen # define what you want to run here: # each task is a list of command/arguments to run, popen style tasks = [['python','httpDatabase.py'],['python','main.py']] # define the file types you want to trigger on # I opted for .py and html files. file_regexp = re.compile("(.py$|.html$)") def files_have_changed(old_stats, new_stats): if len(old_stats) != len(new_stats): return True for k in old_stats: if new_stats[k] != old_stats[k]: return True return False def get_stats(): stats = {} f = [] for root, folders, files in os.walk(os.getcwd()): f.extend([os.path.join(root,x) for x in files if file_regexp.search(x)]) for file in f: try: stats[file] = time.localtime(os.stat(file)[8]) except: pass return stats handles = [] def stop_current(): if len(handles) > 0: for h in handles: print "Killing %d" % (h.pid) os.kill(h.pid, signal.SIGTERM) del handles[:] def restart(): print "Files changed. Restarting" stop_current() for t in tasks: p = Popen(t) print "Started %s. PID:%d" % (" ".join(t),p.pid) handles.append(p) last_stats = {} while 1: current_stats = get_stats() if (files_have_changed(last_stats, current_stats)): last_stats = current_stats restart() time.sleep(1)