Musings++

Something something programming

Anachronistic Programming

I've decided to repost this article that I originally posted on September 17, 2013 while cleaning up the blog archives from when I was using Fargo.

I want to show you a piece of code. Something that's touted whenever this language is spoken of, and everyone seems to be able to pull out of their ass. I want you to look at it, understand it and then see how far down the execution chain you can take it. I'm not talking about if you can debug the app, set a breakpoint and then step through it. I want you to sit there, hands off the keyboard and go through the request cycle that needs to occur for this particular piece of code to execute and run as you expect it to. Spare no detail.

Did you get far enough? Did you get down to the HTTP protocol? The packet dance that happens before the actual request from a browser is sent? Or did you stop at "Browser makes a request to the IP address"?

See the problem today isn't that the computer is this magic box that only a few can understand. It isn't relegated to the guys in beards and thick glasses. It isn't just for the geeks and nerds. To be a programmer meant that you needed to also understand the specific hardware stack that you were working on. The exact chipset, the exact instructions available to you. The exact specs on the memory and video controllers.

Today, computers have become common place. And in order for it to get to this stage a few things needed to happen. The first one being "It just works". That's the basis of the consumer computer. With no fiddling, no worrying about any kind of internals, you should be able to get up and running in no time.

But you're not just a typical consumer. You're a programmer. And unfortunately, this idea of "It just works" has found its way into programmers minds everywhere. You don't need to think about the HTTP protocol, "It just works". You don't need to worry about Little vs Big Endian - "It just works".

Until it doesn't.

I think the problem with programmers today is quite simple. We've been lead to believe that we can rely on certain things within the system. Which, is great. I mean, if we couldn't rely on the HTTP protocol where would we be today? But this reliance has led to an entirely new problem - "I don't care". Programmers today don't need to learn about how the protocol works because "It just works". They don't need to think about it. And I think that has lead to an entire generation of programmers who don't understand the fundamentals of programming. They don't understand that the code they type into their fancy IDE's is really powered by the ideas of a few people and run on hardware. There's a severe disconnect between hardware and software and that is hindering them without knowing it.

Don't get me wrong, I'm not trying to say that I'm some incredible programmer - far from it. I'm actually a terrible programmer, because programming isn't all software and algorithms. There's a hardware component to it that's overlooked way too often. The things you're doing with code you're RELYING on the hardware to accomplish.

Don't you think you should at least have a vague understanding of how it works?


Becoming a web developer

The web is a big deal. Like, a HUGE deal. And the people that make the web have been thrown from their basements into the limelight. They've been lauded and applauded for being on the forefront of the technological innovation. But everything has its cost. Jonathon Hill posted today about the cost of being a web developer. I think Jonathon missed an important cost - he forgot what it's like to be starting out. It's a common problem, and one that inevitably plagues even the great ones. I don't mean to belittle what Mr. Hill does for a living. While I don't have a lot of experience with his work, I'm sure he's a great web developer. And I'm sure that his work is incredible because of the tools he has.

What I take issue with is that he seems to believe that this is what a new developer requires to be great. On the contrary, I think having tools like this at your disposal from the beginning causes one of two things.

  1. You start, the tools don't confer god-like web development skills. You get upset and leave.
  2. You start, the tools don't confer god-like web development skills. You get upset and work harder.

One of those two is good - and that same one doesn't require the initial investment that Mr. Hill thinks.

The real cost

  • Laptop (Hey, you probably have one of these right now!) ~ 700$
  • Books ~500$*
  • Linux (this one is optional, but if you're new to development I'd recommend it. There are a lot of great tools and utilities out there that show up on linux first. They won't be pretty, but god dammit they'll be awesome.)

*optional

Total Startup Cost: ~1200$

The books, of course, are not required but eventually you'll find that there are some things that people will always refer to that you'd like to have around. Javascript: The Good Parts for example. Or the Dragon Book. Books are very important.

The beauty of the web is that it isn't memory intensive when you're trying to figure out what you're doing. But it can grow to whatever you want.

What about training?

Technology moves fast. Really fast. You know that awesome new phone you got two months ago? Out of date. You know that great new framework you learned last year? Technology has made that irrelevant. I will agree with Mr. Hill to a certain degree here. A technology focused college education is quite the waste of time. However, I don't recommend skipping it right away. There are other things that a college education offers you apart from your program.

Presentations, working with others, taking charge of projects when you end up with a bunch of slackers. Making the tough decisions to kick that one dude out of your group because he does nothing. Essays, reports, being on time. And the most important to a FREELANCE WEB DEVELOPER THAT MR HILL SEEMS TO MISS. MANAGING YOUR TIME. For many people College is the first time when they're left to their own devices. They are responsible for themselves and they have to figure out how they work best, and how to manage their social lives AND their work lives. People are give 4 years to make this work. Four years when you're allowed to screw things up and start over. Because the thing is, once those 4 years are up, if you don't have some understanding of how to be you - you're pretty fucked.

However, don't waste your education on a technological degree. Instead, I'd recommend doing something unrelated. Psychology, Marketing, or even English/Theater.

See there's a weird stereotype about a lot of tech people - they tend to be rather introverted. This isn't true for everyone of course, but for those whom it is, you have to understand that even when you enter the workforce as a developer, you still need to interact with people. A LOT.

You have meetings and phone calls, you have to explain your choices to management and clients. If you're a freelancer, you have even MORE work. You need to be a sales guy, support staff and a developer. If you find it hard to talk to people - good luck.

It's a great time to be a good developer.
~ Jonathon Hill

How true it is Mr. Hill. It is a great time to be a good developer. But starting out with a 3000$ investment doesn't make you a good developer.

Having the drive to be better makes you a good developer.


Again, I feel like I have to point out - I think Mr. Hill has some great work. Browsing through his projects, I'm not claiming he doesn't know what he's talking about. Just that maybe he's forgotten what it's like to start out as a developer.


Git core.autocrlf

Linux is my development environment of choice. It wasn't always - I used to do all of my work with XAMPP and Windows, but eventually I got sick of waiting for cool things and just made the jump. Now I can't imagine actually getting any work done NOT in a terminal.

Vagrant and Virtual Box allowed me to have my linux environment for work and my Windows one for gaming. However, it means I get to run into a few issues I never have before - namely constant End-of-Line issues with git.

Windows uses the CRLF (Carriage return/Line Feed, /r/n) ending for lines, whereas unix uses just the LF (Line Feed, /n). This generally means that there's a whole mess of ^M characters in vim due to some files having the dos ending and some being unix-y. Git will try and correct this automatically but for the longest time I had no idea how to actually set that up.

core.autocrlf

Git has a configuration setting called core.autocrlf that tells git how you want it to handle line endings. There are three options:

  • true
  • false
  • input

true

Essentially turns on autocrlf which means that if you check something in/out it will preserve the line endings that the system expects. IE: On Windows it will have CRLF endings, on unix it'll have LF.

false

Turns it off (obviously). This essentailly leaves you alone.

input

Converts everything to unix-y endings.

In the end I settled on git config core.autocrlf true for my case. It modifies the line endings for dos based files to unix-y and then leaves me alone. Exactly the kind of option I'm looking for.


Making vim your IDE

I freaking love vim. Every so often I spend a couple weeks checking out another IDE to make sure I'm not missing anything and then I inevitably end up back at vim. Over time I've assembled a few plugins and workflow that make vim perfect for me.

pathogen

Pathogen is a plugin manager for vim that utilizes git to keep plugins organized. Once you add pathogen to your vim install, you can easily add new plugins simply by cloning the repo. There isn't much to say about pathogen except that it is the greatest.

NERDTree + NERDTreeTabs

The first ime you pop open vim you'll notice that you can't really browse around files. You need to know the filename that you're attempting to open, as well as the directory structure. While you can get around this with some tmux file listing, it would be a heck of a lot easier if it just had a file listing like a normal IDE. NERDTree does just that - a list of directories and files that you can navigate through with your keyboard.

NERDTreeTabs is an addon for NERDTree that allows you to have the same NERDTree instance open across multiple vim tabs.

tagbar

Tagbar is something that I don't actually use that often - however it's one that I'm always glad is around when I need it. It gives you the code strucutre of a file. Classes, methods and variables are all listed allowing you to easily jump to definitions and get an overview for what content is in a file.

Normally when I open a file I haven't been to in a while I'll pop open the tagbar and quickly glance at the methods to re-acquaint myself with things before I close it and get back to work. You can also jump straight to definitions by navigating around the tagbar, but I rarely end up doing that. There is a weird issue with having multiple buffers open simultaneously (I use a lot of vertical splits with vim) where you can't navigate to the tagbar as it displays the tags of the last split that you were in. Therefore I can't really navigate over to the tagbar from a split that isn't directly next to it. Technically I could move the splits around, but I feel like that's more trouble than it's worth.

ctrlp

Apart from NERDTree, ctrlp is the other way that I navigate files in vim. Often times I know the file that I want to work on.. but it's buried within a huge directory structure that I don't really feel like navigating or typing out. Instead ctrlp performs a fuzzy search over the current directory and allows me to open the file within a few keystrokes.

tmux

My development environment has always been linux - namely virtual box/vagrant which I SSH in to. This means I get a straight terminal view with minimal colors and no real "windows". In order to do a bunch of things at once, I use tmux which is a terminal multiplexer. Essentailly it allows me to replicate having multiple terminal tabs/windows within the same session. It even has the added benefit that if I get disconnected from the server I can reconnect and just re-attach my previous session. While tmux isn't actually a plugin for vim, I find using vim with tmux is a great deal better than just vim alone. I get to have a tmux window for code, for git, for mysql and for logs.

Install tmux: sudo apt-get install tmux

dotfiles

Now that I've kinda got my development environment customized how I like, I want to have it with me no matter where I am. I develop on multiple vms on two different machines, as well as multiple remote machines. I'd prefer if my environment was always the same. In order to ensure that I can get up and running as soon as possible on any new machine I run the following commands:

bash sudo apt-get install git tmux curl
mdkir ~/personal/
cd !$
git clone http://github.com/angelor/dotfiles
cd dotfiles
./setup

There aren't that many customizations in my .tmux.conf file and my .vimrc ones. Mostly it's to set up plugins and to remap some keys for tmux.


API Logging with Graylog2 - PHP Logging

This is Part 2 of the API Logging with Graylog2 series. View Part 1

Now that you have the backend components configured for logging, it's time to set up and configure GELF - the Graylog Extended Log Format.

Step 1: Composer

GELF support for PHP is only available via Composer. The installation instructures are pretty straight forward, so I won't attempt to go into too much detail - Composer does an excellent job of covering the basics. Composer Installation Instructions.

Once that's done and set up, you'll need to set up your composer.json file as follows:

{
    "require": {
        "graylog2/gelf-php": "0.1.*"
    }
}

Then just run composer install or composer update if you already had a composer.json file.

What this will do is grab the gelf-php libs and toss it into a ./vendor/ directory wherever the composer.json file exists. It will also configure an auto-loader so that you don't have to figure out what files to include.

Step 2: Log

Now that we've got the library where we want it, we can go ahead and start the logging!

$transport = new Gelf\Transport\UdpTransport('127.0.0.1');
$publisher = new Gelf\Publisher();
$publisher->addTransport($transport);

$message = new Gelf\Message();
$message->setShortMessage('some log message')
        ->setLevel(6);

$publisher->publish($message);

That's all there is to logging to Graylog2. However, there are a lot more things that you can add to your message to give your log a bit more substance.

Customizing your message attributes

One of the things that isn't really documented very well (with the library at least) is what exactly can constitute the message. The Message object includes a few additional methods that you can use to get the most out of Graylog2.

  • setShortMessage - Just a short descriptive message about the log.
  • setFullMessage - This is where you could include any backtraces or additional dumps.
  • setLevel - The "severity" of the log. It follows the standard syslog levels
  • setAdditional - This method accepts two args, the first being a custom key, the second being the value. This is a neat way of adding new information to your log (API keys for example).

Personally, I think Graylog2 is a phenomenal way to achieve a proper logging system - something that is often overlooked when you're in "app dev" phase. I've talked about planning when I talked about Lines of code as a metric and logging is definitely one of the most easily overlooked features - that's super easy to add from the beginning. Logging provides you, not just a way to track errors, but also track progress. Imagine tracking your API usage with Graylog2 and watching the requests/hour steadily rise. And then, because you thought about logging from the beginning, you can easily display the additional attribute "api_key" and "execution_time" that you've been logging to keep a better eye on your server.


Lines of Code

Lately there's been talk again about "Lines of Code" as a metric for progress on a project. The idea is, developers are being graded based on how many lines of code are checked in on a daily/weekly timeline. Unfortunately, not only is this unbelievably stupid, it's also not "new" stupid. It's the kind of stupid that's just been around for a while.

In early 1982, the Lisa software team was trying to buckle down for the big push to ship the software within the next six months. Some of the managers decided that it would be a good idea to track the progress of each individual engineer in terms of the amount of code that they wrote from week to week. They devised a form that each engineer was required to submit every Friday, which included a field for the number of lines of code that were written that week.

Bill Atkinson, the author of Quickdraw and the main user interface designer, who was by far the most important Lisa implementor, thought that lines of code was a silly measure of software productivity. He thought his goal was to write as small and fast a program as possible, and that the lines of code metric only encouraged writing sloppy, bloated, broken code.

He recently was working on optimizing Quickdraw's region calculation machinery, and had completely rewritten the region engine using a simpler, more general algorithm which, after some tweaking, made region operations almost six times faster. As a by-product, the rewrite also saved around 2,000 lines of code.

He was just putting the finishing touches on the optimization when it was time to fill out the management form for the first time. When he got to the lines of code part, he thought about it for a second, and then wrote in the number: -2000.

I'm not sure how the managers reacted to that, but I do know that after a couple more weeks, they stopped asking Bill to fill out the form, and he gladly complied.

Interestingly enough, this silly idea is back yet again. A recent post over on /r/programming talks about a teacher who grades based on LoC.

Senior developers write less code

Here's the thing. I believe that the newer you are to programming the more time you spend programming. Think about your early projects - you had an idea and you just jumped right into it didn't you? You knew exactly what you wanted to do, exactly what features you wanted, and you just hacked at it until it was done. You hit that Builders High. When you're new to programming you chase that high as ferverently as possible. You've never tasted anything quite so sweet.

Unfortunately, the time eventually comes to make some updates to your software. And you think "You know what, if I just re-write it with what I know now.. it'll be like 10x better!". And off you set yet again, chasing that high. The thing is, eventually you get sick of building the same thing over and over again and you eventually decide to just build it smart the first time.

So you sit down and plan out the requirements, you organize the database, and you organize your folder structure. You set up npm, Grunt and LESS. And before you know it, your done your day without a single line of code being written. But you feel productive - because you're no longer a code-monkey hacking out LoC.

Not a builder, an architect

See the difference between a Jr. dev and a Sr. dev is simple. Forethought. It's the act of sitting down and actually figuring out the best way to do something. To plan and research methods before deciding on what's best for the situation. You're no longer just building software. Now you're architecting a solution. You have to see the vision at the end, and then imagine all the little pieces and how they'll work together. As a Sr. developer you won't spend as much time writing code as you will planning code. You'll spend a heck of a lot of time planning. And you'll love it because you know it's the right way to do it.

Just the other day I was speaking to another developer about some new features they were planning on adding to their site. Turns out the software was designed with the eventuality of adding these features. Actually implementing them required a handful of changes and some testing time as opposed to the weeks it was originally estimated that it would take to augment the current application with the new features from scratch.

That's what the Sr. Developers best asset is. Forethought.

See you're not paying a Sr. Developer for how many lines of code they're writing. You're paying them for all the lines of code that no one needs to write.