7 Jun

MailGun as an SMTP Server for Django Apps

Posted by godavemon in Uncategorized.

Gmail is a frequently used SMTP host for Django apps.  Its free and relatively simple to setup which is ideal for small apps.  There are a few significant downfalls. First, its limited to 500 emails a day with no option to upgrade.  Second, it limits and even re-writes all outgoing emails to the username of the connector.

For example if you’re trying to send an email from welcome@yourhost.com or support@yourhost.com you’re out of luck because all of those emails will actually get sent at dave@yourhost.com or whatever email account you have configured. I’m sure this saves millions of people from millions of spam but it adds a bit of annoyance to developer’s lives.

The recently launched MailGunoffers a great solution.  Its free for up to 200 emails per day and super cheap and dramatically scalable after that.  It also works as a drop in replacement for whatever service you were using for your django smtp server.  Changing will take you less time than reading this article.

The Steps:

  1. Sign up for MailGun
  2. Go to the control panel and click on the YOURHOST.mailgun.org server created for you.
  3. In the upper right you will find your “SMTP Authentication” credentials for this server.
  4. Open the settings.py in your django app and configure your email with the given credentials.  It should look something like this.
    EMAIL_USE_TLS = True
EMAIL_HOST = 'smtp.mailgun.org'
EMAIL_HOST_USER = 'postmaster@YOURHOST.mailgun.org'
EMAIL_HOST_PASSWORD = 'SOMEPASSWORD'
EMAIL_PORT = 587
  5. Test it out!./manage.py shell
    >>> from django.core.mail import send_mail
>>> send_mail('MailGun works great!', 'It really really does.', 'tester@YOURHOST.com', ['YOUREMAIL@gmail.com'], fail_silently=False)



							

		


	
		

			

				5
				Jan
			

			
The PickledObjectField for Object Storage in Django

			

			Posted by godavemon in databases, django.						

			

				
I’ve become a really big fan of the PickledObjectField provided by this django snippet.  So much so that I use it in almost every django model I create these days.


Basically it serves as the best way to do an object store in your database and perfectly translates in any JSON conversion.  Its an essential tool in any javascript heavy application.


In my current project, Chart.io we’re aspiring to be google analytics for your database.  We’re basically creating a system that will handle all of your dashboard/analytics needs in the easiest way possible.  This means that we have a LOT of different charts and more in the making.


Its not feasible to put all of those extra parameters that each chart type requires into the model as different columns.  We would end up with an incredible mess in short order.  So I instead create a PickledObjectField called ‘params’ in the model.


class Chart(models.Model):
    ...
    params = PickledObjectField(default={'just': 'some', 'default': 'parameters'} )



The params variable then takes most any dictionary of parameters and automatically converts it to string to be stored in the database.


The following command for example will save a params value of something like “KGRwMQpWa2V……” to the database, but you can still use it just like any dict object.


>>> chart = Chart(params = {"type": "scatter", "dot_size":, 4, "color_list": ["red", "green", "orange"], });
>>> chart.save()
>>> chart.parms
{"type": "scatter", "dot_size":, 4, "color_list": ["red", "green", "orange"], }
#You can also treat the field just like a dict
>>> chart.parms['awesome'] = 'for sure'




Its worth noting that there is a similar snippet to this that uses JSON object to string conversion instead of Pickle.  I find that when using Javascript so heavily its easier to use some other string conversion so as not to get confused and I’ve been really impressed with the way that this particular snippet works.


Its incredibly rare that a django snippet becomes such a major tool.  With the exception of my subdomain middleware, I can’t think of another snippet that I use more regularly which leads me to think that it should really get moved into the core fields that django provides.  Object store is essential element to many applications and the PickledObjectField is the best way to do it.


							

		


	
		

			

				1
				Dec
			

			
On Productivity, Organization and Tools

			

			Posted by godavemon in thingsilearned.						

			

				
I’ve been a fanatic about finding the best ways to organize my life and stay “on top of things”.  There are a million articles online on how to go about this. I’ve read my fare share of these and tried a lot of the methods out.  Usually something ends up working decent for a while and then I either burn out or lose discipline in it.  So I pick up some other method or just deal with a more chaotic productivity schedule.


I never get too disappointed when one doesn’t work out as usually said method was proposed by some blogger who seemed to only have recently (and likely temporarily) gotten their shit together and decided to tell the world about it.  So there wasn’t too much of an expectation going in, but its often a bit frustrating to me to have the advice and tips given out so confidently.


The articles are usually written in a way that makes their method sound like the writer has been practicing this for years and achieved great success/peace with the workflow they deal with.  But I have never met a person who has found such success.


There are a thousand different TODO apps on the web and for download.  Each one will tell you that their tool is the best to keep organized and productive.  There are even more authors and bloggers.


In the end nothing has worked near as well for me as a blank notebook and a pen.  Sometimes a planner helps when my days are more about scheduled meetings and less about development time, but planners are just paper with a different set of lines.


So I guess the thing that I learned hear is that keeping organized and on top of things is a constant struggle and there’s no perfect fix for it.  Its just the nature of the game, and if you accept that it takes time and some effort then the whole process becomes a lot less frustrating and a lot less time is wasted trying out so many different tools.


							

		


	
		

			

				27
				Jan
			

			
Dynamically Calling Fancybox from Javascript

			

			Posted by godavemon in javascript, jQuery.			Tagged: , , .			

			

				
Fancybox is a really nice jQuery library for creating lightboxes.  There are a few simple examples and some documentation on their site but there seems to be no demonstrated method for loading a fancybox dynamically with javascript.  All of their examples require you to embed a link to the lightbox content somewhere in the page source.


The following hack, mostly taken from the method described here, allows you to dynamically any web page into a fancybox straight from javascript.


I’ve documented the following example which should be fairly straight forward.  Basically it embeds a hidden link, that’s controlled by the callFancyBox function.  Make sure you have the right things included and you can simply call


callFancyBox( <your iframed url > );



<html>
<head>

<!--- These are the scripts and styles needed for fancybox to work --->
<script type="text/javascript" src="http://qwisk.com/sbmedia/scripts/libraries/jquery-1.3.2-mod.js" charset="utf-8"></script>
<script type="text/javascript" src="http://qwisk.com/sbmedia/scripts/libraries/jquery.fancybox-1.2.6.js" charset="utf-8"></script>
<link href="http://qwisk.com/sbmedia/css/jquery.fancybox-1.2.6.css" rel="stylesheet" type="text/css" media="all" />

</head>
<body>

<h3> Fancybox Test</h3>

A fancybox will load in 3 seconds

<!--- Currently fancybox only works on links in your page.  This div hides a link, who's href we change dynamically --->
<div id="hidden_clicker" style="display:none">
<a id="hiddenclicker" href="http://asdf.com" >Hidden Clicker</a>
</div>

<script type="text/javascript" >

// Register all links with-flash as fancybox's
$(document).ready(function() {
 $("a.overlay-flash").fancybox({
 'padding': 0,
 'zoomOpacity': true,
 'zoomSpeedIn': 500,
 'zoomSpeedOut': 500,
 'overlayOpacity': 0.75,
 'frameWidth': 530,
 'frameHeight': 400,
 'hideOnContentClick': false
 });
});

// This function allows you to call the fancy box from javascript
// origional source: http://outburst.jloop.com/2009/08/06/call-fancybox-from-flash/
function callBoxFancy(my_href) {
var j1 = document.getElementById("hiddenclicker");
j1.href = my_href;
$('#hiddenclicker').trigger('click');
}

// Call the Fancy Box 3 seconds after the page loads
$(document).ready(function() {
 window.setTimeout("callFancyBox('http://google.com');", 3000)
});
</script>

</body>
</html>




							

		


	
		

			

				11
				Jan
			

			
When Can I Stop Saving for Retirement?

			

			Posted by godavemon in finance, life.			Tagged: .			

			

				
I learned some interesting things about financing retirement yesterday.


I’ve been reading a financial book a friend recommended to me preaching the common methods on “how to get rich the safe way” through managing your 401(k) and IRA’s and such.  It’s somewhat helpful in recommending what banks to use and laying out all of the details of each of the methods of saving and recommending a path.


At the beginning of the book there’s this table similar to many others I’ve seen.  It looks like this:







Smart Sally
Dumb Dan




When beginning to invest the person is
25 years old
35 years old




Each person invests $100 for…
10 years
30 years




At 8% interest at 65 they have
$200,061
$149,036






The basic gist is, starting young is incredibly beneficial.  If the goal was to get to 200k Smart Sally could just stop saving for retirement at the age of 35 where Dumb Dan, at the same rate spent 30 years trying to make it and never did.


When can I be done?


Which got me thinking: what would I have to do to one day just be done saving for retirement?  I wrote a script to determine just that.  It ignores a few minor details which are probably negligible anyway when compared to the error margins in the assumption of inflation and assumed investment return, but gives good ballpark results.


I assume I’d like to retire on an income of today’s equivalent of 50k after tax which is completely generated by the interest earned on my retirement savings.  I assume I can maintain an 8% average earnings and account for a 3% inflation rate.


Without Further delays, here are the results.


Welcome to the Reverse Financial Calculator!

$50,000.00/year is $158,351.35/year when you are 65 assuming 3% inflation
To make that much money from an 8.00% interest you need $1,979,391.86 

You can stop saving up if you reach a total savings of one of the
values in the STOP AT at the corresponding AGE.  If you want to hit
that goal starting from 0 you need to START PAYING the listed value per year.

AGE     STOP AT       START PAYING
26       $98,402.32  $98,402.32/year
27      $106,274.50  $51,093.51/year
28      $114,776.46  $35,355.00/year
29      $123,958.58  $27,508.99/year
30      $133,875.26  $22,819.90/year
31      $144,585.28  $19,709.20/year
32      $156,152.11  $17,500.34/year
33      $168,644.28  $15,855.05/year
34      $182,135.82  $14,585.38/year
35      $196,706.68  $13,578.56/year
36      $212,443.22  $12,762.81/year
37      $229,438.68  $12,090.27/year
38      $247,793.77  $11,527.81/year
39      $267,617.27  $11,051.75/year
40      $289,026.65  $10,644.72/year
41      $312,148.79  $10,293.69/year
42      $337,120.69  $9,988.69/year
43      $364,090.34  $9,721.98/year
44      $393,217.57  $9,487.41/year
45      $424,674.98  $9,280.09/year
46      $458,648.97  $9,096.04/year
47      $495,340.89  $8,932.02/year
48      $534,968.16  $8,785.34/year
49      $577,765.62  $8,653.75/year
50      $623,986.87  $8,535.38/year
51      $673,905.82  $8,428.63/year
52      $727,818.28  $8,332.13/year
53      $786,043.74  $8,244.74/year
54      $848,927.24  $8,165.44/year
55      $916,841.42  $8,093.36/year
56      $990,188.74  $8,027.74/year
57     $1,069,403.83  $7,967.93/year
58     $1,154,956.14  $7,913.33/year
59     $1,247,352.63  $7,863.45/year
60     $1,347,140.84  $7,817.81/year
61     $1,454,912.11  $7,776.03/year
62     $1,571,305.08  $7,737.74/year
63     $1,697,009.49  $7,702.62/year
64     $1,832,770.24  $7,670.38/year
65     $1,979,391.86  $7,640.77/year


Pretty cool right?  The results say that if I pay about 20k a year I can stop putting money away for retirement in just 6 years, making my last payment when I’m 31.


When to Start


I also was interested in seeing what it would take to do that same 6 year goal if I put off starting the plan.    You can see from the following results that putting it off does have significant cost to it.


START      STOP    PAYMENTS
26           31    $19,709.20/year
27           32    $20,665.96/year
28           33    $21,669.16/year
29           34    $22,721.06/year
30           36    $23,824.02/year


So the plan would cost about 1k more per year for every year I put it off.  That’s a lot, but it doesn’t necessarily mean you should put everything aside and go out and start saving ASAP.  As long as whatever you’re doing, that’s not allowing you to put significant money away is increasing your probable future income by atleast 3k (post tax) per year you should be in good shape.


People who work towards a PhD or an for example MD are not able to save much but will be able to handle that extra 4k/year easily once they’ve got a much higher paying job (schooling aside).


Anyone interested in trying it out themselves or modifying the results slightly can grab the source here.


Fun Fact:




If your parents had gone cheap with your baby clothes, crib and stroller and instead put 10k a year away from the day you were born, you’d be set for retiring at 65 on just your 2nd birthday!


AGE    STOP AT
0       $28,691.65  $28,691.65/year
1       $30,986.98  $14,897.59/year
2       $33,465.94  $10,308.63/year


							

		


	
		

			

				7
				Jan
			

			
Comparing Github and BitBucket

			

			Posted by godavemon in revision control.			Tagged: .			

			

				
There are a lot of advantages to using a distributed revision control system as opposed to centralized.  There are also some down sides, mostly in that it can be a bit more confusing to manage.  A centralized web interface however seems to solve that problem quite nicely.


Currently there are two major competitors in that space, Github and Bitbucket, that manage Git and Mercurial repositories respectively.  I’ve used Github for a while and recently have really started using Bitbucket and I thought I’d share some thoughts and comparisons of the two.  There are a lot of blogs that compare Git and Mercurial, but this is not one of those blogs.  I personally have a preference for mercurial, but for admittedly stupid reasons like it was written in python and the syntax more closely resembles SVN which I’ve become used to.  To me, a major differentiator in choosing one over the other is the interface or ‘hub’ that manages it.


Below is my item by item lineup showing the differences between GitHub and Bitbucket







GitHub
BitBucket




pull requests
yes
yes




forking
yes
yes




command instructions
yes
yes




distributed revision control
yes
yes




network graph
yes
no




inline editing
yes
no




developer profiles
yes
yes




wiki
yes
yes




issues
yes
yes




rss feeds
yes
yes




source browser
yes
yes




monthly price
cheap
cheaper




free private projects
0
1




# of users
lots
not lots




happy hours
in SF
no






There really aren’t a lot of differences.  Personally, I actively use both.  Learning both is easy as the syntax difference between the two revision control systems is minimal and both hub interfaces are very similar.


I use git for open projects like TwitTornado where privacy is not an issue and where I’d like to get more developers involved. Its really got a fantastic social network of hackers.


I use bitbucket for my private repositories as it allows 1 free private repository (per user) and has cheaper monthly rates.


I hope this helps others make a decision between the two and try both of them out.  Let me know in the comments if I’ve left any parameters out of the table above and I’ll add them in.


							

		


	
		

			

				30
				Sep
			

			
Initialize Django Tests

			

			Posted by godavemon in django, python.			Tagged: , .			

			

				
I use a lot of doctests for apps that all need to work on a set of initialized data.  I was hoping that there would be some kind of hook in Django for this but there is not.


I could switch all of the doctests to unittests and use fixtures but that would be a lot of work and I prefer doctests.  I could also go through and paste some sort of init command at the beginning of each test that would ensure the data was loaded or do the loading but that’s just plain bad practice.


I came up with a method of creating a ‘testsetup’ app that is always run before the other apps ensuring that whatever that app configures or loads into the database will be run first before any other apps preform their tests.  Here’s how you can do it too.


First create a ‘testsetup’ app and edit its tests.py file


./manage.py startapp testsetup
open testsetup/tests.py



__test__ = {"initialize tests": """

>>> your init code here

""" }




In the test.py file you can load the database, prime the cache, or setup whatever else you need initialized.  Then add the ‘testapp’ as the first app to your settings.py



INSTALLED_APPS = (

'testsetup',

...

)




Now whenever you run


./manage.py test


It will first run the tests for the ‘testssetup’ script and everything will be primed.  If that’s the only kind of test you run then that’s all you’ll need.  You’re done with this tutorial.


But if you run app level tests (ie. ./manage.py test someapp anotherapp ) then the above solution is not enough.  To ensure the testsuite is run before these apps we’ll make our own TEST_RUNNER.  Create a file called ‘testrunner.py’ with the following source.



def run_tests(test_labels, verbosity=1, interactive = True, extra_tests=[]):
print "Given these test_labels", test_labels
print "With these extra_test", extra_tests

from django.test.simple import run_tests as django_run_tests
if test_labels:
# Make sure 'testsetup' is run first
tl = ['testsetup']
tl.extend(list(test_labels))
test_labels = tuple(tl)
print "Testing these apps:", test_labels

django_run_tests(test_labels, verbosity, interactive, extra_tests)




and then in your settings.py file set the TEST_RUNNER variable



TEST_RUNNER = 'testrunner.run_tests'




This script simply wraps the django test runner to ensure that the testsetup app is tested before any other apps are.  It basically makes the django test runner think that you’re running ./manage.py test testsetup someapp when you actually run ./manage.py test someapp.


Proposal


A note to the community: I think it’d be great if the Django settings.py included a TEST_INIT variable which allowed you to point to a function that would be executed immediately before the first test was run.  The hook would make the setup process for doctests much easier.


							

		


	
		

			

				26
				Aug
			

			
Forwarding Naked Domains for Appengine with Apache

			

			Posted by godavemon in Apache, appengine.			Tagged: , .			

			

				
Google Appengine currently does not allow configuration of naked domains.  Meaning, if you’re building something on Appengine you’ll have to settle for a URL like http://www.mysite.com or http://whatever.mysite.com and you will not be able to use http://mysite.com.


Its not so bad in most cases, but not having http://mysite.com isn’t going to stop people from trying to go there.  So its important to set up some sort of device to forward the naked domain to the www.


Unfortunately you’re going to need a server and for this example it will need to be running Apache2.  The redirect is handled by placing the following line in an apache config file (/apache2/httpd.conf).


Redirect permanent / http://www.mysite.com/


Where http://www.mysite.com is the example site being hosted on Appengine.


Or, if your apache server is hosting other apps and domains you’ll need to set up the redirect in a VirtualHost as shown here.


<VirtualHost *:80>
ServerName mysite.com
 Redirect permanent / http://www.mysite.com/
</VirtualHost>


Note that the ‘permanent’ parameter in the Redirect command enforces a 301 or permanent redirect.  You can choose other forms of redirect by replacing the ‘permanent’ with either the redirect number (ie 302) or the keywords from the following table I’ve shamelessly copied from here.






HTTP Code
Status
Description




301
permanent
The resource has permanently moved




302
temp
The resource has temporarily moved




303
seeother
The resource has been replaced and refer to new resource




305
UseProxy
Use proxy to access site




307
Temp
The resource has temporarily moved




410
Tegone
The resource has permanently removed






Finally set your DNS to point the base domain to this apache server and in a few hours it should be permanently re-directing http://mysite.com/ to http://www.mysite.com/.


If you’re not using apache or are looking for more ideas here’s a list of other techniques.


Also, please vote for Google to fix the issue here and here.


							

		


	
		

			

				11
				Aug
			

			
Socialbrowse in the StarTribune

			

			Posted by godavemon in Uncategorized.						

			

				
Yesterday I was on the cover of the Business section of the StarTribune.  Its a nice article that talks a bit about Socialbrowse and my path to what I’m doing right now.


It also talks about Luke Francl and his awesome startup FanChatter which is also from Minnesota and also in YCombinator.  Here is a link to the article


http://www.startribune.com/business/52719012.html


The print version comes with pretty pictures of Luke and I which I think is worth the money :) .


Thanks Thomas Lee for the nice coverage.


							

		


	
		

			

				10
				Aug
			

			
Log Observer using Python’s subprocess

			

			Posted by godavemon in python, thingsilearned.			Tagged: , .			

			

				
Today I needed to write a wrapper around a program that would examine the stdio and respond immediately to certain results.  The task was handled nicely with a python script using subprocess.


There isn’t a lot of documentation examples on it so to figure it out I wrote this small test app and I thought I’d share it here.


To begin I wrote a simple script that prints out “Program has run for x seconds.” every x seconds to simulate a long running and noisy script that I might be observing.  Here is the code.


sample.py


import sys, time

class FlushFile(object):
   """Write-only flushing wrapper for file-type objects."""
   def __init__(self, f):
       self.f = f
   def write(self, x):
       self.f.write(x)
       self.f.flush()

# Replace stdout with an automatically flushing version
sys.stdout = FlushFile(sys.__stdout__)

for i in xrange(100):
   sys.stdout.write("Program has run for %d seconds.\n" % i)
   time.sleep(1)



The main part of the program is the last 3 lines where we write “Program has run for %d seconds” 100 times and pause for a second between each.


The FlushFile object is simply a nice hack to overwrite the default stdout object to ensure that the buffer is flushed every time it is written to.  Without the hack our log checker would simply be hung up until the task is fully completed instead of being able to read each line of the output as it is written.


The output looks like this




$ python sample.py
Program has run for 0 seconds.
Program has run for 1 seconds.
Program has run for 2 seconds.
.... # and on until 100 seconds




Next I wrote the wrapper which executes the above sample.py script as a python subprocess and watches the output.  For this example I check whether a ’4′ exists in the output.  If it is I restart the script therefore creating a continual loop counting between 0 and 4 seconds.


observer.py



import subprocess, os, signal
cmnd = "python sample.py" #change this line to run your script
p, line = True, 'start'

while True:
   p = subprocess.Popen(cmnd, shell=True, stdout=subprocess.PIPE)

   while line:
       line = p.stdout.readline()
       print "line is:", line
       if line.count('4'):
           print "restarting the process"
           os.kill(p.pid, signal.SIGUSR1)
           line = True
           break

   del p



The output is continually checked in the ‘while line:’ loop of line 8.  In this example its printed out for our convenience.  Like 11 checks for the character ’4′ somewhere in the output.  If it does exist it kills the process and the stdout reading loop.


That loop is wrapped in another however that simply restarts the process all over again.  The resulting output is




$ python observer.py
line is: Program has run for 0 seconds.
line is: Program has run for 1 seconds.
line is: Program has run for 2 seconds.
line is: Program has run for 3 seconds.
line is: Program has run for 4 seconds.
restarting the process
line is: Program has run for 0 seconds.
line is: Program has run for 1 seconds.
line is: Program has run for 2 seconds.
line is: Program has run for 3 seconds.
line is: Program has run for 4 seconds.
restarting the process
line is: Program has run for 0 seconds.
.....




So now we have a nice template for wrapping and responding to executables!


							

		


	
		

			
« Older Entries

			

		























	

	

			
	

		Follow
		

			
					

		
			

			
Get every new post delivered to your Inbox.