Karthik Hariharan

Reflections on Software

Beating the Talent Crunch With a Distributed Team

There’s been a lot of press lately about the severe shortage of software engineers across the country. I have been involved in hiring engineers in multiple companies over the past few years and I’ve seen a steady shrinking and dispersion of the available talent pool. I believe that the only solution to the talent crunch in the short term is to build and grow distributed teams.

After working closely with successful distributed teams in the past, I became aware of how well they work when grown carefully. To that end, I sought out a role with a distributed team to learn how they operate and utilize these skills to help build and lead teams in the future.

This post is to share some of what I’ve observed and learned after 6 months of working in a distributed team at Pure Charity.

Distributed Hiring

Hiring members of a distributed team is not all that different from hiring a colocated team. Here are some core guidelines.

  • Focus on internal referrals from active team members.
  • If your internal referrals run dry, reach out to the larger community.
  • Look at actual code via OSS contributions or code snippets they share.
  • Spend a day pair programming remotely on a real world problem to see how they fit in with the existing team.
  • Try to keep everyone within a single timezone.

Everyone knows a weak link hurts the whole team, but in a distributed team, it can hurt even more. The good news is, you have a wider talent pool to work with and can afford to be a bit pickier with whom you hire to mitigate this risk.

The Chatroom

The team chatroom provides a number of advantages. Here are a few.

  • A common communication area where team members can get rapid feedback and bounce ideas around.
  • A focused area for private messaging without the noise of typical IM networks.
  • A watercooler for the random banter that builds camaraderie among teams.
  • Relief from the constant flood of email.

The last point is very important. Since working in a distributed team, my email volume has dropped significantly and I don’t have to spend much time managing my inbox to stay on top. That has been invaluable.

There a number of products, aside from the standard IRC, that can host the chatroom. My favorite is HipChat, but many teams use Campfire. I like the hosted services since they can parse code, share files, embed images/video, and log conversations. Plus, they are a lot of fun when you add a Hubot.

Audio Conferencing

When ideas are not easily communicated via IM, email, or the Chatroom, it’s best to hold an audio conversation. There are a number of solutions to this but I suggest the following guidelines:

  • Choose one that runs on low bandwidth networks with good audio quality.
  • Everyone on the team should use the same product.
  • Have everyone stay online as much as possible during the work day.

Our team, being composed entirely of Mac users, uses iChat to quickly have push button audio communication within small groups. Many other teams use Skype. Choose what you like, but make sure everyone uses the same thing and it remains frictionless.

Video Conferencing

A robust video conferencing solution is essential to help team members hold both regular meetings and ad-hoc touch points. The things to look for here are:

  • Good cross platform support.
  • Excellent audio and video quality for multiple participants.
  • The ability to quickly let participants share their screen.

My favorite tool for this is GoToMeeting, but many teams use WebEx, Skype, or free products for this. The better solutions come at a premium price, but it’s well worth it. Regular face time over video conferencing helps the team gel better.

Remote Pair Programming

A great way to help development teams stay focused and productive is to give them the right tools for pair programming. In our team, nearly everyone is comfortable using Vim as their editor. With the combination of Tmux + Vim, team members can remotely pair program and control a single shared terminal session. In this termimal session, everyone has keyboard control on a low bandwidth connection. This is extremely useful when pair programming or mentoring more junior developers.

If your development environment really requires the use of an IDE, I have heard good things about using TeamViewer as an alternative.

The Workstream

This suggestion may be a little controversial, but I suggest having every team member keep a running log of what they work on during the day. When you’re working in a distributed team, it’s easy to lose track of what each team member is working on. While you can always IM or make calls, it’s far less intrusive to glance at the workstream to get a heartbeat on the team’s progress.

In addition, the workstream provides the whole team with transparency and helps keep everyone focused. You can more easily see if someone is spinning their wheels and help them get unstuck long before they have to ask. This helps a lot with onboarding and ramping up new team members without making them feel too micro-managed. Also, it helps build an environment of mutual trust and understanding.

We use Coopapp and Harvest to have team members track their work hours along with any paid time off.

If tracking time feels a little too intrusive to you, I recommend using Yammer and having your team provide micro-updates periodically.

Don’t Cross the Streams

One of the biggest areas where I see companies go wrong is mixing remote team members with colocated members. Nothing sucks more than being the one guy on the phone in a room full of colocated developers. It can be isolating, frustrating, and terribly unproductive for everyone involved.

If you’ve already got a colocated team, the best thing you can do is hire more colocated team members. If you’re struggling to hire locally, then hire two or more remote workers that have worked together previously to seed a distributed team. Keep their work separate from that of your colocated team. Or let some of your colocated team members work from home to help seed the distributed team.

Plan for Face Time

At least once a quarter, plan on getting everyone together in one physical location for a few days of work, along with a night or two out. Such sessions will be highly effective and the evenings are great for team building. During the day, set a clear agenda and try to stick to it.

Final Thoughts

Distributed teams definitely come with certain challenges, and it’s critical to keep the teams small to minimize the communication difficulties. Many companies like 37signals, GitHub, and Living Social have all used these strategies effectively to build a remote workforce so there’s no reason it can’t work for you too. If you have thoughts or experiences to share, please do so in the comments.

An Open Source Workflow With Ruby on Rails

I’ve been an advocate of open source for a long time. First via software that I used, like Mozilla’s Firefox, and later though consuming open source code and contributing back to it. I’ve used open source code on a number of platforms over the years including PHP, Java, .NET, and Ruby on Rails.

Out of the platforms that I have used, I believe Rails has the most accesible open source ecosystem because it enables consumption and contribution better than most other platforms.

This post will be my attempt at describing a developer’s work flow when using open source with Ruby on Rails.

Discovery

The first step to consuming open source to use on a Rails project is discover existing open source libraries, called gems in Ruby, to solve a problem that you encounter while building your application.

There are number of screencast sites, podcasts, and mailing lists out there that highlight useful gems and how to use them. Here’s a few that I use:

Most gems are hosted via RubyGems.org. This site also maintains a search index with links to source and documentation for any gems hosted there.

GitHub is also a good place to discover gems via search.

Some key criteria that I look for in a gem that I’m considering for use in a production application are the following:

  • Publically Hosted on GitHub
  • Recent commits within the last 3 months
  • A high number of Forks/Watchers
  • A low number of currently open Pull Requests.

These are just my personal guidelines, so feel free to relax them as you see fit. For side projects to hack on, you can (and should) be a lot more willing to try more experimental gems.

Installing Gems with Bundler

Once you’ve identified a few gems that you’d like to use, the standard recommendation is to use Bundler to manage your external gems.

Bunder is a dependency management gem that uses a manifest file called a GemFile that lists out the gems in the root of your project.

To add a dependency in the simplest way, edit your Gemfile and add a line similar the following line to add the Carrier Wave gem.

GemFile for Carrier Wave
1
gem 'carrierwave'

Once you’ve updated your Gemfile, run ‘bundle install’ to have Bundler pull down your gems and install them for use in your app.

Adjusting your versions

In the course of using your gem, you may find that you need to use a different version than the current version published on RubyGems.org.

To go back to an older published version of the gem, you can adjust your Gemfile like below.

GemFile for Carrier Wave with version
1
gem 'carrierwave', '0.5.4'

After updating your GemFile, run ‘bundle update ’ to force Bundler to reinstall the gem using the correct version.

If you’re feeling more adventurous, or simply want to test the most current code for the gem with your app, you can point directly to its public git repository by putting this in your GemFile.

GemFile with git repo here.
1
gem 'carrierwave', :git => 'git://github.com/jnicklas/carrierwave.git'

Finally, if you’ve cloned the gem’s source code locally, you can point to a local directory for the gem here.

GemFile with local directory here.
1
gem 'carrierwave', :path => "~/Projects/carrierwave/"

Forking to fix

If you find that the gem you are using contains a bug, and the gem’s code is on GitHub, it’s very easy to attempt a bug fix and test it locally in your application.

The first step is to fork the gem’s repository using your GitHub account.

Then clone your fork locally.

1
2
cd ~/Projects
git clone git@github.com:hkarthik/carrierwave.git

Next, point your application to your locally cloned Git repository for your fork of the gem which contains the bug.

GemFile with cloned local directory here.
1
gem 'carrierwave', :path => "~/Projects/carrierwave/"

Now you’re all set to edit your local clone of the gem and fix it.

If you’re using your application to test your fix, you’ll need to run ‘bundle update ’ to rebundle the gem into your application whenever you make changes.

Because this can sometimes be a slow process, I advise writing tests within the gem’s source code first before bundling into your app for integration testing.

Once you’ve fixed the gem, you can commit your fix, and push to your fork on GitHub.

1
2
git commit -am 'Fixing this gnarly bug'
git push origin master

Then you can modify your application’s GemFile and point to your GitHub fork of the gem:

GemFile with git repo of forked gem
1
gem 'carrierwave', :git => 'git://github.com/hkarthik/carrierwave.git'

From here, you could continue to maintain your fork and keep it up to date with the latest changes from the original gem’s repository. While you can certainly choose to go this route, I would recommend contributing back the fix with the eventual goal of deleting your fork.

Contributing back

So now that you’ve fixed the code in the gem, you’re back to cruising along on your app. Why not make the fix available for anyone else using the gem? GitHub makes this process easy.

Browse to your fork on GitHub and click the Pull Request button.

Fill in some details about the fix you’ve made. Submit the pull request and wait for feedback from the project maintainers. If they accept your pull request then your fix will make it into the master repository instantly.

If your fix is accepted and later incorporated in the next published version of the gem, I recommend moving your application back to the published gem and deleting your fork on GitHub.

While you can leave your fork out there, if you’re not actively maintaining it, you’re likely to confuse anyone else looking for the source code of the gem. Unmaintained forks are just unnecessary noise on GitHub, and I don’t suggest keeping them around.

Conclusion

I hope this post gave you a good start on how to contribute to open source while building applications in Ruby on Rails.

You don’t have to be an expert to start contributing to open source. All you need is the right tools and a little bit of knowledge on how to get started. It’s a great feeling to watch your first pull requests get accepted into your favorite open source project and realize that you’ve just given back.

I’ve used a lot of open source in the past, but only recently have I felt confident enough to start contributing back. I’m glad I did, as it’s great feeling to give back in some small way to projects that have saved me countless hours while building applications. I hope that I’ve inspired you to do the same.

My Reasons for Leaving .NET

My current gig at Pure Charity is my first developer role where the technical environment is not .NET.

After 6 years of .NET development work, I’ve decided to leave .NET and work in Ruby. My reasons for making this change were largely technical but also reflected a change in attitude about building software.

This post will be an in depth explanation of my reasons for leaving .NET.

Back to Unix

Working with Ruby on a day to day basis has rekindled my love for Unix.

The philosophy of Unix is usually summarized as:

Write programs that do one thing and do it well.

This concept really identifies with how I think about software that I write and what makes my favorite software tools so great.

In college, most of my programming work was done on on Unix based systems. While much has stayed the same on the Unix side in the last decade, my toolset has changed slightly. Where I used to use Solaris/Linux, Emacs, and Bash, these days I use OSX, Vim, and Zsh, respectively.

Coming back to Unix was a big change from the Visual Studio focused Windows development environment that I had used for the past 6 years. But after a few months, I feel more productive than ever.

Embracing Git

Git has become one of my favorite software tools. While some may think of it as just source control, for me it represents a different approach to distributed work, experimentation, and simplicity around source code.

Combined with the use of GitHub as a collaborative tool, Git has simplified my entire workflow around writing code, reviewing other people’s code, and utilizing Open Source.

Most of the Enterprise tools that the vast majority of .NET shops use to accomplish these things come with far too much friction for my tastes. This friction leads to lots of wasted time, more bugs, and lots of general frustration among the Agile teams that I’ve worked with in the past.

Switching to the Mac

I purchased my first Mac in 2009 and I have found the hardware and the software on the Mac to be simple, elegant, and easy to use.

The single biggest benefit to the Mac is that beneath the flashy hood, it’s a powerful Unix-based OS and can utilize all the tooling and libraries that Unix developers have used for decades. Coupled with the power of Unix and the ability to run familar software with the GUI or through virtualization, there’s little reason for any developer to not consider moving to the Mac.

Another benefit that I have found with the Mac is that it requires far less baby-sitting of the Operating System in the way of driver updates, OS updates, and general system administration. By simplifying all of these tasks, the Mac has allowed me to focus more on consuming and creating content rather than constantly tweaking my OS or worrying about the latest update.

While some enjoy the tweaking, as I did in the past, I no longer enjoy spending my time that way. I now prefer my OS to stay out of my way so I can focus on perfecting my craft and building things.

Deploying to the Linux based Cloud

Many companies today are starting to move away from self hosted infrastructure and are instead deploying to Amazon EC2 or to other cloud based hosting providers. Such public clouds are built on top of the virtualization layers and foundations provided by Linux.

Unix based Operating Systems provide far more system level configuration, setup, and automation via the shell. These types of administrative tasks most often are done via GUIs in Windows, making things more inefficient by requiring more dedicated personnel to manage environments.

With Azure and other efforts, Microsoft is making efforts to address these short comings with Windows Server. But to me it makes little sense to pay exorbitant license fees for immature and less elegant solutions to manage cloud-based server infrastructure.

Joining the Ruby Community

I’ve always admired the Ruby community and the hacker focused culture around it. Open Source is at the heart of this community and reputations are built upon significant open source contributions. You’d be hard pressed to find a Rubyist speaker who doesn’t put their code out there for the world to see and critique. This makes the whole community feel more genuine.

The Ruby conference talks are more often in a show and tell format. Rarely do I feel that the speaker is attempting to sell me on something or further their personal branding. The .NET community tends to have a strong vendor focus along with lots of self promotion at the speaker level. It’s been refreshing to see less of this in the Ruby community.

The Ruby community also feels more self sufficient than the .NET community, since Rubyists don’t look to a vendor to direct their technology stack. Many .NET developers will not approach a technology unless there is a Microsoft flavor of said technology, even if it is the poorer choice compared to mature open source offerings. There are countless examples of this over the last decade, and I expect to see countless more.

Working at Startups

After a few years of working with Enterprise Software companies, I realized that I’m better suited to work with startups. I prefer new product development over legacy software maintenance, and it’s very difficult to find such opportunities inside traditional Enterprises without political maneuvering and a lot of luck.

In a startup, there is usually a lot more room for an individual to experiment and take ownership of an idea without getting too deep into internal politics and bureaucracy.

Unfortunately, startups are less likely to use .NET technologies due to its higher costs. To be fair, Microsoft has made efforts to reduce costs for startups through the Bizspark program. But after 3 years, a startup is still subject to very high licensing costs and has to strategically plan to pay for them or migrate off .NET. Given the choice, I don’t know many startup founders that would choose .NET for new ventures today.

Additionally, there is a perception that .NET developers are too specialized to be effective generalists. Since I specialized in .NET for the past 6 years, this severely limited my options when seeking to work with startups.

Specialization is for Insects

My journey away from Windows and .NET has been around 2 years in the making, beginning with my purchasing a Mac and making a concerted effort to build a product using Ruby on Rails.

It’s been a long and hard road to get here, but I received a lot of support from my friends and former coworkers who similarly left .NET recently. For their help, I am eternally grateful.

I’ve definitely had to make sacrifices to get here, including a loss of seniority, but I’m confident that I made the right choice and that these sacrifices will be well worth it in the long run.

In the future, rather than specialize solely in Ruby, I’ve chosen to grow more as a generalist developer. In the coming years, I’m looking to tackle other technologies like Node.js, Scala, Clojure, iOS, and Android.

I don’t regret my time in .NET, as I made lots of wonderful friends and got the chance to work with some very smart folks. But it’s time to move on to other things. I look forward to sharing more of my journey with you.

A Fresh Start on a Static Site

I’ve decided to move my blog to a new domain and use a static site generator to serve up the content. Since the content doesn’t change much, it makes little sense to continue using Wordpress on a shared web hosting plan with limited resources.

Static Sites Are Fast

A static site significantly reduces the overhead of disk I/O by taking the relational database out of the picture. Most modern web servers can be highly tuned to avoid disk access for static content by utilizing output caching. If setup correctly, the fast majority of web requests require zero disk I/O. This allows the web server to serve content faster and handle larger loads with less resources.

While it is possible to use Wordpress Plugins like SuperCache to reduce disk I/O, it still requires a Fast CGI process to serve the content via PHP. This incurs enough CPU usage to make it slower than a pure static site.

Easy cloud based hosting and scaling

While a typical shared hosting setup works most of the time, it does carry the possibility of a popular post bringing your hosting to its knees. Planning for this kind of event carries a lot of cost which will go unused most of the time.

Cloud based hosting platforms, like Heroku, are perfectly suited for hosting static sites that can be scaled up very easily. Heroku’s free plan goes a long way and would likely handle the vast majority of traffic, but still give you the ability to use push button scaling for those rare instances when a post hits the #1 page of Hacker News.

Choosing a static site generator

There a number of static site generators that will perform the basic operations of translating posts that are written in Markdowninto HTML and organizing the output along with any uploaded media.

I chose to go with Octopress because it offered the following:

  • Ruby based and easily customizable with plugins.
  • Easily configured to work with Heroku.
  • Stylesheets are written in Sass.
  • Excellent typography out of the box with the base theme.
  • Sits on top of Jekyll, and can utilize many existing plugins.
  • Easy local preview with a built in Rack server.

Migrating old content

I’ve chosen to move my old wordpress install over to wordpress.com’s free plan for archival purposes rather than attempt a content migration. But if you want to preserve your old content and links, there is a Wordpress to Jekyll migrating tool called ExitWP that many have used successfully. Also, you can utilize a rack server to execute code to handle redirects.

Additional Resources