Requirements for a Text Editor

These are my minimum requirements for a competent editor. For each of these points you should have multiple tricks up your sleeve to make your editor the best choice for you. If your editor is falling short try and address a bullet point at a time to improve your workflow.

  1. Navigate with ease (jump to line number, bottom/top of file, selection/deletion helpers [inside block, function, etc])
  2. Fuzzy-find filename in project structure
  3. Run spec(s) without having to Alt+Tab
  4. Find Regex pattern within file (replace, count, etc)
  5. Search entire project (git grep or Ag inside the editor)
  6. Organize code into tabs or windows for clarity
  7. View git changes within file (+/- on line numbers) and basic git workflow integration (git blame, commit/push without leaving editor)

Replacing Heroku

For anyone still on Heroku and throwing addons at your application please stop and seriously reconsider what you are doing. Across the board things are getting a bit ridiculous for the average hobbyist / fincially conscious company. I used to be quite the Heroku fanboy but after my recent experience with Hosted Graphite I have changed my tone completely.

I was recently looking to add some Graphite metrics to a Heroku application that I’m working on and stumbled across a solution called Hosted Graphite. There was of course a starter package which shows off the beauty of Grafana and a few metrics to give you an idea of the potential. The current pricing can be found here (this prices do fluctuate quite regularly depending on the service); I’ve also gone ahead and added the current prices as of writing this below:

Each price point adds a few key features to the mix + increases your metric limit (this is a bullshit number stored by whatever service you are using so feel free to push it without too much fear. I’ll do a follow-up post on my experience with a MongoDB solution that had a document “limit”). The paid features for Hosted Graphite are as follows:

  • Daily Backups (>Tiny)
  • Data Retention (>Tiny)
  • Hosted StatsD (>Small)
  • Account Sharing (>Medium)

For most folks that are looking to use Graphite in their application they are probably looking to use some StatsD wrapper to report metrics. If you only have a few data points that you care about and can deal with the low retention rate / backup policy then maybe you could add an aditional worker to handle the task and not have to deal with a “hosted StatsD” solution. Even with that stretch of the imagination you are looking at a $19 monthly solution but most likely going to be suckered into a higher cost if you seriously start to add metrics.

Rather than paying for a plugable addon I’d highly recommend spinning up your own Digitalocean server for $10 bucks and get as much value as the Small package (this is being very generous..) being offered on Heroku. If you don’t know how to spin up a Graphite+StatsD server there is no excuse with the resources provided by DO alone:

Assuming that you can get through a basic set of steps then you should have a working Graphite+StatsD service at your disposal with ½ the monthly cost. The interface will be exactly the same (API key/namespace + service endpoint) and you will have the freedom to manage the server as you please. And when you are measuring things you sure as hell don’t want things to stop reporting because a reporting mechanism fails and you have no way to tell..

This is another major gripe that I have with Hosted Graphite; why would I pay for a service like “hosted statsD” when it will fall over and I have no debug info into why that might have happened. When testing with a Small instance of Hosted Graphite the StatsD endpoint would become unresponsive for minutes at a time while UDP/TCP were working as expected. On the other hand with a droplet you have the freedom of config (and Graphite, Carbon, and StatsD have a lot of configuration).

Configuration brings me to the final point I want to make. “Features” like Data retention length are nothing more than configuration that can be managed when setting up your Graphite instance. By configuring carbon you can set custom retention policies on different data points (regex matching FTW) and ensure that you have the policy in place that makes sense for the data being stored. This gives you the freedom of full flexibility inside of a problem-set that you are actively engaged in when adding measurements (naming new metrics). As a best practice you should set a reasonable default and add new metrics into the appropriate policies as needed.

Deploying Jekyll Site on Ubuntu VPS

Over the past few months Heroku has become an increasingly expensive option for projects that are starting to scale in any way (Addon price hikes, Dyno Sleeping and Charging, etc.). For ease of use and speed to spin-up up a prototype and have it running Heroku is still the leader and will continue to be my goto in times of need. But as soon as Dyno Charging becomes an issue I think it’s time to capify that app and use one of the many VPS hosting options out there.

For this project I wanted to migrate my simple static site to a $5 droplet on DigitalOcean. I’m sure you could get a better price on AWS but am fine paying the little bit extra for ease of use on DO.. After going through the initial server setup (groups, perms) it’s time to go though the actual server preperation.

Server Preperation

sudo apt-get update
sudo apt-get install nginx

Ensure that nginx is properly installed by visiting the public ip from the following command

ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'

If everything went correctly you should see the following nginx welcome screen (nginx will automatically start when installing on Ubuntu so there is no need to start the service before running the test):

For basic nginx service management use the following commands:

sudo service nginx restart
sudo service nginx stop
sudo service nginx start

Now that we have nginx ready to go on our server let’s install the final pieces to bring our Jekyll site to life.

sudo apt-get install git-core
curl -L | bash -s stable --ruby=2.1.2

After installing rvm there will be some information about the install process and some next steps that may be required to start developing. If you do not have access to rvm on the command line after install you may have to run the following or add to your ~/bash_profile:

source /usr/local/rvm/scripts/rvm

Once Ruby is running correctly go ahead and install jekyll and all it’s glorious dependencies.

gem install jekyll

Server Setup Compete

Now it’s time to set up your jekyll site for deploy. For this example we are going to use Capistrano 2.x

gem install capistrano
capify .

This will create a few config files but the one we want to concern ourselves with is config/deploy.rb

set :application, "blog"
set :repository, 'public'
set :scm, :none
set :deploy_via, :copy
set :copy_compression, :gzip
set :use_sudo, false

set :user, "deployer"

# the path to deploy to on your VPS
set :deploy_to, "/home/#{user}/blog"

# the ip address of your VPS
role :web, "XX.XX.XX.XXX"

before 'deploy:update', 'deploy:update_jekyll'

namespace :deploy do
  [:start, :stop, :restart, :finalize_update].each do |t|
    desc "#{t} task is a no-op with jekyll"
    task t, :roles => :app do ; end

  desc 'Run jekyll to update site before uploading'
  task :update_jekyll do
    # clear existing _site
    # build site using jekyll
    # remove Capistrano stuff from build
     %x(rm -rf public/* && rake generate)

Modify a few lines to suit your needs and you should be ready to cap deploy. To verify your new Capistrano deploy run the following; if both commands succeed without Warnings or Errors you will ready to deploy.

cap deploy:setup
cap deploy:check

Enjoy your Jekyll VPS

Matchadd vs Match

The target of this post is to clear up any confusion about the subtle differences between matchadd and match/2match/3match. This is not meant for the novice vim user but folks that have some experience with vimL. I’ll try to keep it as simple as possible for anyone interested in some of vim’s inner workings.

From :help match we get the following information:

:mat[ch] {group} /{pattern}/
        Define a pattern to highlight in the current window.  It will
        be highlighted with {group}.  Example:
            :highlight MyGroup ctermbg=green guibg=green
            :match MyGroup /TODO/
        Instead of // any character can be used to mark the start and
        end of the {pattern}.  Watch out for using special characters,
        such as '"' and '|'.

        {group} must exist at the moment this command is executed.

        The {group} highlighting still applies when a character is
        to be highlighted for 'hlsearch', as the highlighting for
        matches is given higher priority than that of 'hlsearch'.
        Syntax highlighting (see 'syntax') is also overruled by

From the definition it should be pretty clear that match is responsible for applying defined highlights to the current window. The {group} argument is simply the name of the highlight group definition you would like to use on a given {pattern}.

To visualize a list of all available highlight groups you can use :highlight. This is really handy when programatically trying to add hightlight groups for a plugin. As you can see from the screenshot below our colorscheme has redefined the majority of the groups and :hi does a great job showing all of the possibile combinations.

Using that information we could use any one of the groups and apply a match in our current buffer. Let’s take a look at using the ErrorMsg highlight group to match the entire contents of a line.

:match ErrorMsg /\%13l/

If you have not used match before take a moment now to play around with it in your own vim session and get a feel for how vim reacts to all different types of scenarios. Which match takes priority when running consecutive matches on the same pattern?

One of the main limitations with using match is that you cannot continuously apply new matches and have the previous ones stick. Because of this it is not the most suitable option for creating any sort of full fledged highlighting plugin. If you read the help section below match you will encounter :2match and :3match which is meant to provide a way to manage 3 seperate highlight groups. Unfortunately these functions suffer from the same shortcoming we saw with the original match.

In order to compensate for the limitation on match you can create more complex patterns to include multiple line selections (/patern1\&pattern2/) or whatever else you would normally do inside of the typical vim search regex (remember a touch of magic is only a \M away with this type of searching). If you’d like to see evidence of the hacks that are possible while using check out the first iteration of vim-poi (fresh-start branch is ready for testing and this logic will all be replaced with the smarter and better matchadd pattern)

  for i in b:poi_lines{a:group}
    let c += 1
    if c == 1
      let s:build_string = s:build_string.'/\%'.string(i["line_num"]).'l\&\M'.i["content"]
      let s:build_string = s:build_string.'\%'.string(i["line_num"]).'l\&\M'.i["content"]
    if c == len(b:poi_lines{a:group})
      let s:build_string = s:build_string.'/'
      let s:build_string = s:build_string.'\|'

Stop Being Lazy and Learn Haml or Another Templating Engine

The desire to write familiar view code, similar to the same html you were writing to support your shitty LAMP apps is completely understandable but it’s time to move on from the glory days..

There seems to be this idea that using erb is “good enough” in all scenarios because so many of web devs are accustomed to seeing their old friend html.. As someone who knowingly opted out of using Haml because HTML always felt like option that would give me the most flexibility. Since the actual engine driving the browser has a simple set of responsibilites (MASSIVE OVERSIMPLIFICATION: fetch a document from a URL and render the contents of that document onto the page) the document that we are going to be providing to the browser’s engine must adhere to standards that allow many browsers to function with their various implementations. Because of this fact debugging/learning a templating engine is actually a walk in the park if your HTML skills are as sharp as they should be..

To be fair; when researching Haml it may not be clear what you are getting yourself into.

<section class=”container”>
  <h1><%= post.title %></h1>
  <h2><%= post.subtitle %></h2>
  <div class=”content”>
    <%= post.content %>

To be honest the above code doesn’t look to be that bad to me but I’ve never really had an “eye for design”. As an additional bonus I know exactly what the browser output is going to be (assuming no CSS + JS messing with things). But let’s take a look at some haml code..

  %h1= post.title
  %h2= post.subtitle
      = post.content

The raw number of keystrokes should be striking to get the same browser output. The declarative simplicity that Haml provides is unbelievable when you consider all the bullshit we deal with when writing HTML. Anyone who written web apps can tell you at least one tail of a missing closed tag.

I was hesitant about writing something that wasn’t what I knew and used for a long time. The notion of “what works for me” is not a good enough reason to write code that is sub par. Since most devs are bright enough folk I don’t think learning Haml is more than a 1 micro-project away from being your new favorite way to write your views.

For more information on Haml check out the Haml Homepage. All links to get started should be easily accessible from there.

The beauty of using haml in every day development is that your code shape will be forced to be much nicer and cleaner. Consider writing a container div with a header, body, and footer. One would hope for clear structure that tells a “foreign eye” exactly what is going on in the given view.

Just imagine the skeleton that would evolve in the html version of this as a clean slate to start from.

<div class="container">
    <div class="header"></div>

    <div class="content">
    </div> <!-- end of content -->

    <div class="footer"><%= render "footer" %></div>

</div> <!-- end container -->

As anything gets added to this skeleton you hope that the structure stays readable. I think that the forced structure prevents you from being your own worst enemy . Because of this simple fact I am turning into a Haml guy and think a weekend using it will convert any non-believers.

Thoughts on Heroku

After deploying over 10 applications with Heroku I think that I can finally make a fair assessment on the general developer / Heroku relationship.

After watching an excellent talk by Matz at Waza (link at the bottom) I was taken aback by how much he seemed to respect Heroku’s ability to support the developer that wants to build. Heroku’s use of AWS to anyone to standup a system to run their code at no cost is an absolutely stunning accomplishment .

My skepticism with this idealistic view is that at the end of the day Heroku is a business that needs to make money. But they also need to keep their image and offer the “playground” to build and prototype without needing a budget.

During the talk Matz mentions this ability to spin up a prototype at no cost which as being something fundamentally new and exciting. Having this ability eliminates a the large risk of money puts people in a new age; one where there is no barrier of hardware costs preventing concepts to come to life.

Heroku as a company proves that hardware is cheap and that it is feasible to remove the hardware barrier that used to exist less than 10 years ago(someone tell me if I’m wrong, I was 13 and it was too expensive for me). For developers that like to build things from the ground up Heroku is a wonderful tool that should be utilized to its fullest for as long as you can use it for free.

I am not sure about the exact price model of Heroku’s production ready systems are but I think that it becomes clear pretty quickly that if you are building to grow and scale Heroku may not be the best option.

I will use a real world example of a small sinatra app that is currently hosted on Heroku for the steep price of $0.

I have 1 Dyno with 512 MB of RAM that likes to shut off when it becomes inactive for a certain period of time (smart on Heroku’s part for sure to keep costs down), 1 MongoHQ instance with 512 MB of storage (they call it sandbox mode so this is an expectedly low amount of storage).

When upgrading from the Standard (free) dyno it would cost me $34.50 to essentially double my computer power and have a cycling set of dynos that will work together as a failsafe mechanism. I have just recently started crashing dynos and am thinking that it would be nice to have a backup that is reliable without having to do any system work to setup proper restarting and recovery (no god debugging for me sounds pretty sweet).

For any additional storage I would be paying around $18 per GB of SSD storage hosted through MongoHQ (of course an Amazon service wrapped in an add-on pricing model). I am not sure what the going rate for a mongoDB hosted solution but that seems a bit pricey to me if you are going to do large lazy data storage like I have been enjoying. But to be fair that could easily be moved to another database since in reality it’s just a minor detail.

When laying out the costs to double the performance of my small application it would be ~$52 a month. TBH that isn’t a horrible price for no extra work scaling. I am curious what the minimal cost of this setup would be on AWS directly (I do know that if you want to run tests on postgres instances it will cost you a pretty penny at the end of the month).


Matz at Heroku’s Waza (2013)