Coding Reflections

1.1. Planning

Software requires planning. It doesn’t need fancy 4-color glossies and video presentations; pianod2 has a crayon block diagram on artist’s newsprint. Despite not being fancy, it did its job: it helped me sketch out how the components would work together before writing code. So when the code was written, it did what was required, and didn’t subsequently need to be changed to accommodate unexpected needs. Effort wasn’t wasted, and the code wasn’t subjected to edit fatigue.

Similarly, it is productive to sketch out details of planned changes before jumping into coding. A spate of audio enhancements on pianod required updating build configuration scripts, where new flexibility lead to the question, “What should this do? Given the choice, do I include more, or less?” Temporarily abandoning the code to write an abstract, I reflected on what the build was supposed to achieve, which lead to desired behavior, understanding configuration goals, and a set of requirements suitable for coding. Had I rushed the coding, build configuration would have made something—but that something may not have been the best fit for end-users.

1.2. Environment

Furthermore, interruptions are terrible to performance. Have you ever walked in on a programmer, busily coding, oblivious to your presence? Or perhaps she acknowledged you, but refused anything beyond that until she finished what she was working on. Either way, it’s clear: we are in another world. Our brains are full of what we’re doing.

In my case, when I’m working on a piece of code, that module is in my thoughts. There’s a set of instructions I’m trying to express in code, and I have a list of things I have to worry about (What if that file I’m opening isn’t found? What if I ask for memory, and there is no more? Is this secure against erroneous or malicious input? Is this approach efficient?) On the sidelines I have thoughts about other software components my code will interact with, the design it will fit into, and other components to be written after the current one.

Being interrupted breaks the flow. Worse, shifting gears to work on something else brings that something else’s ideas into the foreground, supplanting the details of what I had been working on. When I return to the original project, it will take some time for all the details and considerations to come back to me. And in the meantime, my productivity is down and the quality of my code is compromised.

1.3. Minimizing Worries

• My healthcare, and whether/how I will survive if I am injured, become ill (physically or mentally, possibly as a function of overwork).
• Interpersonal drama
• Corporate drama: reorganizations, layoffs, changes to compensation or in health plans
• Unethical business practices and/or product quality/safety issues

1.4. “Simmering”: Contemplation and reflection

Without preparation, the first approach that comes to mind is used. If it is short-sighted, it will have to be fixed after the fact. If the path taken is a blind alley, then the time and effort invested will be for naught.

Contemplation before coding reminds me of making bread. You assemble the ingredients, mix them up, then set it aside and wait. “I thought you were making bread?” Yes, the dough is rising. “No, it’s just sitting there on the counter with a towel over it.” Yes. “Don’t you have to do something to it?” Keep it warm, but not too warm. “Why not just cook it now?” No! That would produce disastrous results.

We understand that despite the appearance of idleness, there’s microbiological activity rising the bread. There’s something akin to this in many creative endeavors, but without yeast to explain the need for pause, contemplation is seen as idleness, and contemplative programmers are seen as lazy or shirkers. It’s unfortunate.

Contemplation before coding is effective. Most people hate household chores because they are an inconvenience, a waste of their time. I don’t see them that way: they are an essential part of spending time away from the machines, shifting into a different mindset—important because it’s not mentally healthy to spend too much time in cyberspace. Shoveling snow affords me exercise; cooking yields good, healthy food; ironing, dishes and laundry (I use a clothesline) provide a chance to think.

Sometimes, I think about my personal to-do list, my relationships or my aspirations. But sometimes I think about my work, and since I’m not in front of a screen I think differently. Sometimes, I think more broadly, a view of the big picture: how will the modules relate? If the data is structured in one way, one algorithm applies; organizing another way means a different algorithm. Which fits better? Which is more extensible for possible future needs? Sometimes my thoughts are very specific: when I write code for this, I will need to consider that or I’ll have problems with yet another thing. Sometimes I reflect on work I’ve done, and catch problems somewhat after-the-fact: did I check such-and-such a condition?

Contemplation offers foresight to choose an optimal approach and avoid bugs before they are written, while reflection gives hindsight to catch others before they get very far. And because bugs become harder (and more expensive) to fix the more entrenched they are in the code, it’s a good investment.

1.5. Rest

I posit the same for mental activity. Some people, posed with a problem, pick the first thing that comes to mind and run with it; if they are right, they achieve results quickly. On the flip-side are those who study the problem and consider the implications; when problems are complicated, this is likely to achieve a correct result more quickly, because the rush-to-solve types often iterate a few times before getting it right.

But all of these people have a limited amount before their thoughts become addled and they need a rest. Rest is a chance for the brain’s engine to cool down, while sleep cleans the mind of past thoughts, just as washing a window cleans it of the dirt and grime and dead bugs of the day past. And getting away from it all—a holiday or vacation—is like changing the oil. Run a car too long without changing the oil, and it breaks down; run a person too long without a vacation, and they break down too. Only people take much more time and money to fix.

For the record, there is also a third class, those that don’t do much. Given a problem, they first bemoan it, then make a fuss about how complicated it is, and finally tackle it at a snail’s pace. Perhaps they don’t need rest, because they’re never “red lining.” Because these individuals often spend more time explaining how difficult their work is, instead of getting work done, managers may erroneously respect this type, thinking they are accomplishing the impossible.

1.6. Other Passions

Yet we all have interests that are engaging and stimulating. These passions vary over time: out of college, for me, it was D&D and gaming. Later I helped run a club and explored my sexuality. That was in turn supplanted by pianod, which was joined by a passion for aerial silk. All through this I maintained an interest in backpacking and long-distance cycling; other interests have come and gone too.

The American expectations for software engineers—at least 40 hours each week, realistically 50 or 60, with only two weeks of vacation—are inadequate to allow exploration of other passions. This is truly unfortunate. One can push off other passions for a time, but not indefinitely. In the long term, one questions what it’s all about. Why am I making software and bothering to earn money if I can’t do the things I want to try?

I believe passion is important in creating software. My best work is motivated by desire to solve an interesting problem by applying my knowledge, skill and talent. Income is a nice perk, and incentive inasmuch as it funds other interests. Yet, done too long without a rest, software is tiring. Done to the exclusion of all other passions, it ruins the fun and I lose the passion to do it—“burnout”.

Earlier I compared a vacation to changing the brain’s engine oil. Exploring other passions is what creates fresh lubricant. If I take time off to read, relax, and attend to things in my life, I return unburdened. But if I’ve done those things, then been off on an adventure, my desire to make software is renewed. While away, my longing for the stimulation of a software challenge builds anew, and I return reinvigorated about software.

As much as I love making software, doing it to the exclusion of all else is no fun at all. If my heart is to stay in it, I need to explore other interests too. A work/life balance must be struck, and while income is important, it is no substitution for life.

1.7. Rushing may not produce results quickly

1.8. Seeming fast is not necessarily actually fast

By comparison, writing code in compiled languages takes longer. Compilers complain when you declared you would return a value, but you didn’t. They kvetch when you use an undeclared variable or method, because that’s an error (usually a typo). Comparing incompatible things? No, you can’t do that. Compilers are fussy about passing correct variable types to functions, and whine when the parameters are wrong. Some languages allow you to provide multiple methods with different parameters, but passing arbitrary parameters—something all scripted languages allow—is impossible or awkward in compiled languages.

Compilers remind me of copy-editors for cookbooks. Download recipes from the ’net and you might get half-way through only to find out you need a special ingredient, or encounter vague or unclear directions about how to perform a step; you’re obligated to throw away your attempt or improvise and see what happens. Maybe it’ll be edible, maybe not. But with a cookbook, the copy-editor makes sure the recipes are formatted consistently, with all the ingredients listed up front, all the measurements specified including units, instructions clear and specific.

Like the copy-editor, the compiler offers proofreading. Compilers are pedantic, and all that futzing with complaints saves time debugging. Sure, in scripting languages you can try out things out early and often, and debug as you go. But writing anything of any complexity in a scripting language requires a lot of debugging. Passing the wrong variable, making a typo, getting parameter order wrong or leaving one off: all these disappear in compiled languages, leaving only logic errors to focus on.

Compilers pay off even more when revising existing functions. Add a parameter? Your code won’t compile until you fix all the uses of that function—whereas in a scripting language, something will happen (usually a runtime error or misbehavior).

I don’t mean to dismiss scripted languages entirely; I love shell scripts, and JavaScript is a fun little language despite its hazards. And Python’s local-by-default variables, named parameters, and parameter count checking make it considerably more palatable than, say, Perl.

But for all their complaints, I love compilers for all the troubles they give me. Because I am always amazed, after I address a dozen or two compiler objections, how often my code works flawlessly, and how quickly debugging goes other times. I attribute that to compiler pickiness.

2.1. Revise, revise, revise

Software is no different; unfortunately, non-programmers rarely understand this. To outsiders, if it works, it’s done. But this is insane: when else does a first draft get published? Authors move through several drafts before publication, and the work is copy-edited prior to printing. Proof copies are printed and corrections made before mass production.

Well-written software undergoes the same rigorous editing. This doesn’t involve throwing away and starting again, but it does mean allowing programmers to:

• Revise or rewrite incoherent sections of code, whether original or a result of edit fatigue.
• Spend time cleaning up (copy-editing) code that’s sloppy
• Refactor algorithms that were poorly implemented. For example, I once encountered a page and a half of code equivalent to:

     bool is15interval(int i) {
                return ((i % 100) % 15) == 0;
        }

2.2. Software by engineering vs. evolution

While natural evolution is potent enough to produce the diversity of life on Earth, it is not reliable: colorblindness, hemophilia, down syndrome, sickle-cell, and numerous other genetic disorders come with evolution. Evolution can do amazing things, but it comes with side-effects.

Because the patchwork approach to software repair resembles evolution, it produces results reminiscent of evolution:

• Problems that won’t “stay fixed”. A change or fix doesn’t work as completely as expected, and requires repeated patches. Sickle-cell provides resistance to malaria, but misshapes blood cells that then jam in capillaries. If this was software, we would “fix” the capillaries, which might create another new problem, which could then be fixed. All these downstream problems are because the original problem, malaria, was fixed with a hack, and those endowed with this hack have shorter lives.

• Vestigial code. I was working on ASP.NET e-commerce websites circa 2012, and one customer had a page that consistently took 25–30 seconds to load. I found code that used the time doing something mysterious, and after some research, determined it was related to an earlier database architecture. The code was no longer necessary, but nobody had removed it after the database change. I deleted it, and load times declined to less than a second. Much like our appendix, it no longer served a purpose, and only made trouble.

2.3. Automatic Testing

pianod2 added herd testing early on. A single shell script builds a release and distributes it to a heterogenous herd of machines, where each one compiles (using differnet compilers/compiler versions) and runs the product through the unit test. Inadequacies of a compiler, platform-specific issues, execution sequence issues in expressions, and mistakes that don’t show up on the development platform but do on others are a few of the problems herd testing catches.

Herd testing also addresses configuration varieties. For example, different herd members use ffmpeg, gstreamer, and AVFoundation–ensuring these variations are tested regularly.

3.1. The Sucky Side of Software Jobs

3.2. Lack of Longevity

But what the neophytes write and what the experienced write aren’t the same. Neophytes don’t understand the implications of different data structures or algorithms for access or memory utilization. They don’t know the libraries extensively, thus reinventing or doing things the hard way. They make more errors, leading to more debugging, which takes longer for lack of experience. “Easy fixes” come with unconsidered side-effects. And the code they write is haphazardly styled and hard to read.

Second graders aren’t novelists. Neither are forth or sixth graders. High-schoolers only turn into writers if they are motivated to practice and develop the craft, and even then only with time. Writing well is more than taking a gander at the Little Brown Handbook, then giving it a go.

Unfortunately, this isn’t the attitude with programming. Non-programmers ask, “Can you write code that goes? If yes, you’re hired.” Then, once hired, they just want code that “goes”; there’s no assessment of quality. Because not being a coder, “it goes” is the only assessment they can make.

When I read code—both my own and other people’s—there’s the objective and the subjective side. Objectively, does it do what it’s supposed to? Subjectively, is it coherently organized? Does it completely handle special cases or errors that might happen? Is the approach effective and straightforward, or is it a Rube Goldberg contraption?

As one who reads code, I can differentiate failings. A sysadmin friend has clever approaches, but his code is unreliable because he never checks completion statuses. An electrical engineer whose code I inherited was reliable, coherently organized, but suffered badly from a cut-and-paste approach; I used his design but shifted to an object-oriented approach, eliminating 80% of the code and reducing memory footprint of the resulting executable while increasing maintainability. Although his style differs, I respect promylop’s libpiano because the code is coherent, clean, reliable. I assess my own code, too: pianod2 is a better design than the original; the worst of the ugliness has been concentrated into the Pandora source/libpiano interface. The User and Users classes are pretty rudimentary; the recently-redesigned filter code shows my C++ skills are maturing.

EnumeratedArray class stands out as an awesome solution to a problem that many have had, which I developed because when I went looking for a solution to array indexing with a scoped enumeration (enum class), it was clear to me those I saw were bad solutions—a conclusion a cookbooking newbie would probably not have made. Like a boss, neophytes would judge a solution by whether it would enable their code to work, not elegance or type safety.

Skills like the ability to judge code and evaluate strategies are valuable, built slowly over years or decades. They should be treasured for leading to well-written software, but that means we need to stop burning our programmers out just when they’re getting good.

3.2.1. On Hardness

I think one of the thing that burns out so many of us is lack of control over the difficulty of problems, especially when managed by non-programmers who have no sense of how difficult it is to program. I’m going to describe different difficulty patterns I’ve encountered as different mountain challenges.

You can’t climb Mount Everest in one go. You need to plan ahead, and take on the mountain in stages, with periods of rest. Failing to do so, you’ll get part way up the mountain before exhaustion or injury will win, and you’ll end up a lifeless corpse in the icy altitudes. Set your programmers up to climb a Mount Everest, and they’ll burn out spectacularly. New programmers to replace them will be left to pick up half-completed code, with debugging not yet begun. Eventually, the whole project fails when management realizes it can never achieve its goals this way.

A partial solution is to break it into stages, with frequent checkpoints. But you can’t keep this up either; El Capitan is a challenging climb. Some people love rock climbing, but if expected to make a career out of it, supplanting social opportunites, a chance to have downtime and rot their brain on some TV or read a good book sometime, and the fun will be lost. When every waking moment is climbing or recuperating before the next one, climbing loses its fun, becoming a slog. Day by day, exhaustion will set in ever deeper until you can’t do it anymore. Throwing super-challenging problems at your programmers one after another may not destroy them as quickly as Everest in One Go, but soon their passion will be lost and exhaustion will again set in.

Switchback Mountain is the opposite problem. It’s too much a walk in the woods; the same, gentle slopes day after day. It’s pretty, and once in a while it’s great, but do it everyday and it gets boring. Where’s the challenge? What have you done that’s exciting, that shows you something new? There are some programmers for whom the monotony of Switchback Mountain is fine; to them, enjoy. I am not one of them. I do enjoy a walk in the woods, from time to time, but not every day. I want—I need—stimulation of new problems, ideas and skills to keep up interest and motivation.

Appalachian Trail

The Appalachian Trail is an example of “good challenge.” It’s not one huge up. It’s not day-after-day up. It’s more than just a walk in the woods, although sometimes it’s that. Other days it’s a walk on a ridge line, or a road walk through a village or past some country farms to get from one section of trail to another. There are assents and decents, some of them challenging but others gently climbing alongside a stream or following a mountain’s contour.

And the long-distance hikers on the AT aren’t hiking every day. They may be on vacation or sabatical to be out there, but it’s okay to “take a zero day” and rest. Hiking the AT isn’t supposed to be a slog; those hikers that turn it into one, get sick of it and give up.

Talented, self-motivated programmers like challenge. It’s an opportunity to stretch our abilities, to find out what we can do. But like climbing a mountain, it’s exhausting, so alternating that with easier work gives a chance to recover, and firm up new knowledge and skills before taking on new ones. And we also need to have our “zero days” where we do things other than programming, because as much as we love doing it, it will lose its attraction if we lose touch with the joy of creation that is coding, or lose sight of the beauty that can be found in good code.

“Zero days” aren’t just doing nothing, either. On my adventures, they tend to be reading days, but sometimes resupply or repair days. During coding projects, my “zero days” tend to be documentation: updating manuals, adding missing comment blocks and reviewing accuracy of existing ones. I may do small-scale refactoring: renaming badly named variables, improving function names, and trying to improve the code’s clarity: all steps in moving the code from a rough draft into a cleaner, more final form. These are important in creating reliable, maintainable software.

4.1. Know the Language & Libraries

I acknowledge the problem that when picking up a new language, wrangling the syntax can be plenty to deal with. Writing savvy, elegant statements is impossible when you only know pidgeon. When that’s the case, grinding out unrefined code helps gain fluency. That can be balanced with Internet searches for ideas; there’s often someone that’s wanted to do the exact same thing and cook-booking can help build style and introduce appropos pieces of library. StackOverflow is great for this.

After attaining comfort writing code that works, one is still not a language expert. With a basic fluency and a memory scaffold to fit information on, (re)reading a good book on the language cover-to-cover fills in language and library details. While studying each topic, recall related, clumsily-written pieces of code, then go back and revise them. This is where one advances from “adequate” to “excellent” code, learning nuances and subtleties of different expressions, better able to express code concisely and precisely.

4.2. Use Better Languages

Classic pianod was written in C because pianobar, its ancestor, was written in C. As I am implementing pianod2 in C++, I feel déjà vu of switching from Pascal to Object Pascal circa 1993; I can express more action in less time, with less code, more easily. The C++ STL library provides all sorts of handy data structures and algorithms so I don’t have to write and debug them. It’s beautiful to declare a vector<something> and just stick things in it, without having to carefully allocate or reallocate blocks of memory, carefully checking references are correct so I don’t leak, lose, or abuse memory.

Choose wisely when selecting a language at the start of a project. It is usually a decision you’re stuck with for a long time.

Consider that although it may be a hassle to learn a new language, if that language is an improvement over the existing, that it will yield easier maintenance to your project’s code for years to come, and will be a skill available for the next project too.

4.3. Language enhancements

     copy_if(service.begin(), service.end(), back_inserter(connections),
                [user] (PianodConnection *conn) { return conn->user == user; });

This isn’t bad, but uses a lambda function which is C++11; previously, you’d have to use a functor:

struct when {
        User *user;
        when (User *u) { user = u };
        bool operator() (PianodConnection *conn) {
                return conn->user == user;
        }
};

But that’s adding up, so unless you wanted to do this a lot, I think it’s clearer to write the ugly loop:

for (PianodService::iterator s = service.begin(); s != service.end; s++) {
        if ((*s)->user == user) {
                connections.push_back ((*s)->user);
        }
}

But in C++11 my preference is to use the new for loop:

for (auto conn : service) {
        if (conn->user == user) {
                connections.push_back (conn);
        }
}

Any language worth using is evolving, both the libraries and the syntax of the language. Language enhancements allow code to be expressed more succinctly, with better error checking by compilers. They may offer new solutions or better or more adaptable ways of approaching problems. Language enhancements should be embraced, not shunned.

4.4. Formatting & Indenting

Although an 80-column line limit made sense when 80-column terminals were commonplace, the limit is arbitrary in today’s windowed environments. indent(1), clang-format(1), and Astyle can quickly reformat to various widths, adjusting code style as they go. Put a pre-commit script on your revision control system to ensure it’s checked in with a consistent manner; developers can configure a key in their IDEs to reformat in their preferred format, with a comfortable line-length for their display. Technology has solved this issue.

Similarly, we are still bent on monospace editors, though there’s no good reason for this anymore: modern IDEs point right at the errors; no longer do we need to repeat :wqmake<enter><esc>?vi<enter><enter>54G37| (inserting the appropriate file and line numbers from the compile output) to find our way to the problem. But I’m digressing.

4.5. Comb vs. Toothbrush braces

Consider:

f()
{
        exit (1);
}

vs…

f() {
        exit (1);
}

The second is preferred, but this is a religious preference.

Consistent use of curly braces is also preferred, especially if using an editor that doesn’t have auto indenting. It’s too easy to have:

bool was_negative = false;
if (x < 0)
        x = -x;
        was_negative = true;
int y = pow (5, x);
return was_negative ? -y : y;

Furthermore, mixing braces/no braces within a single if is ugly:

enum { negative, zero, positive } x_state;
if (x > 0) 
        x_state = positive;
else if (x < 0) {
        x_state = negative
        x = -x;
} else
        x_state = zero;

Be consistent.

5.1. assert()

Those other pieces of code are often patched to fix problems caused by the contract violation, but as they are deeper the checks may not be as straightforward. After a while code has redundant, obfuscated error checks all over and still isn’t reliable.

Where applicable, return values from functions can also be checked.

5.2. Additional checking

For calls within a library or program, parameter sanity checking is a judgment call based on the function’s contract with calling functions.

Consider, however, that CPU is cheap, debugging is hard, your time is valuable, and adding a check now may save hours later.

6.1. Code should not resemble line noise

test_parser ("shorter_with_optional_and_longer-a", parser, "fish dinner", 1,
             0, NULL,
             1, NULL,
             CHECK_NEXT_FIELD, NULL,
             CHECK_END) || (success = false);

The goal should not be to fit as much action as possible onto each and every line of code. The goal is to write coherent, understandable code that works reliably. Clever abuses of syntax, complicated run-on statements, these are bad things.

If something is getting too complex, you should:

• Move logic to a function, to break up the complexity into manageable chunks, or

• Comment the code to explain what it is trying to achieve, and if complicated, how it is doing it. You don’t need to comment every line, and don’t just write in a comment what the code says.

  Bad:

  for (auto const &item : mycollection) {
          total += item.second.rating; // Add rating to total
  }
  float rating = total / float (mycollection.size()); // divide
  

  Add, divide: yeah, I knew that. Why? Here’s a better option:

  // Get the average rating
  for (auto const &item : mycollection) {
          total += item.second.rating;
  }
  float rating = total / float (mycollection.size());
  

6.2. Guard ifs vs. nested ifs

g() {
        if (a = alloc(object_a)) {
                ...
                if (b = open_file (file_b)) {
                        ...
                        if (c = alloc (object_c)) {
                                ...
                                if (d = calloc (object_d)) {
                                        ...
                                        return true
                                } else {
                                        alert ("alloc object_d botched");
                                }
                                free (c);
                        } else {
                                alert ("alloc object_c botched");
                        }
                        close_file (b);
                } else {
                        alert ("open file_b failed");
                }
                free (a)
        } else {
                alert ("alloc object_a botched");
        }
        return false;
        } /* End of g()

Versus:

g() {
        if (!(a = alloc (object_a))) {
                alert ("alloc object_a botched");
                return false;
        }
...
        if (!(b = open_file (file_b))) {
                alert ("open_file file_b failed");
                free (a);
                return false;
        }
        ...
        if (!c = alloc (object_c))) {
                alert ("alloc object_c botched");
                close_file (b);
                free (a);
                return false;
        }
        ...
        if (!d = alloc (object_d))) {
                alert ("alloc object_d botched");
                close_file (b);
                free (a);
                return false;
        }
        ...
        return true;
}

Suppose a change requires adding another allocation between allocating A and opening file B.

• With guard ifs, the repeated releases of all the previously allocated resources get long and repetitive.
• When changes are necessary, there is a possibility (likelihood?) of forgetting the resource release in some the guard-style cases. This leak is hard to find.
• Guard ifs can work well if there’s no leak-risk work being done, such as parameter checking.
• Nested ifs are preferred for allocations, file opens, or any tasks that require some sort of release/undo if they fail.
• If success requires releasing resources too, consider storing the result in a variable and return that at end of function to take advantage of the error handling releases.

In C, it is up to us to release all resources, and nested ifs are pragmatic to ensure releases balance allocations. In a language like C++, where a variable that goes out of scope is automatically destructed via RAII, guard ifs may be clearer. That all said, the best choice depends on the particular goal, and there is no hard-and-fast rule that applies to all situations in any language.

Incidentally, the above guard if code leaks C if D allocation fails. Did you notice that?

6.3. Case & Type

And perhaps more important than any particular style, is adhering to the style of existing code when making revisions. Conformity to existing style generally overrides any of the following preferences.

• Enumerations and constants are ALL_CAPITALS. C++ enum classes (whose members must be referenced by EnumClassName::MemberName) are CamelCaseWithLeadingCapitals.
• Typedefs, classes, and structs as public classes are CamelCaseWithLeadingCapital.
• C structures (non-class structs) are lower_case; their corresponding typedef is CamelCase.
• Member functions are camelCaseWithLeadingSmall.
• Non-member functions are all_lowercase (C modules) or camelCaseWithLeadingSmall (C++ modules).
• Variables are all_lowercase. This includes locals, globals, members and parameters.