DRY Code, WET Comms

The principle of DRY code is probably one of the most important bedrocks of professional programming. It’s a hallmark of what separates an amateur coder from a legitimate engineer: thinking ahead; designing your codebase; making it flexible, modular, and maintainable.

If you haven’t come across the acronym before, it stands for Don’t Repeat Yourself, and it means that pretty much any time you find yourself duplicating blocks of code or logic, you should think about pullling it out into its own function or class or whatever, that would then be called from both of those places.

xkcd #2347

Why? Because when you later find a bug in it, you can just fix it in one place, instead of fixing it in one place and forgetting to fix it in the other. Or worse, not knowing there even is a second place to fix it in, because you didn’t write the code in the first place, or you did, but it’s been more than three days and you forgot.

Being DRY is the opposite of copying and pasting the same for loop all over the place, because you’re just learning how to program and you’re not sure how to import modules, or really even what a function signature is. That style of programming is WET, because you Write Everything Twice. It’s amateurish and it’s something that a lot of good engineers learn to avoid on their own, after having to fix everything twice, or refactor everything twice, or three times, or six times.

However, people are not CPUs. In fact, we built CPUs, because we’re so bad at being CPUs. For us, seeing the same thing more than once is boring. It’s something we filter out, because boring is not dangerous. The motionless green grass won’t kill us, and so we hardly even notice it. But if it moves in a funny way, suddenly we think it’s because of a snake, and it’s the only thing we can see. We are neural networks. We are built to recognize patterns, to flag any dangerous ones, and to recognize them with high urgency. And so, we are not having fun adding a long list of numbers. Or cross-referencing scientific texts. At least not most of us.

But there’s a tendency, especially in the technical manager, to still think DRYly. “From now on, we’re going to use RabbitMQ for all asynchronous messaging, unless there’s a good reason not to”, you write in the the #dev-general Slack channel, adding that it’s already part of the deployment anyway, that it’s very performant, that it usually doesn’t make sense to reinvent the wheel, and that it’ll be easy to adapt existing code to use it. When you’re done typing this pithy paragraph, you tag it with @channel, giddily dismissing warnings about how many people in what timezones will be getting dinged. “Good! Ding them!”, you cackle. You handle the handful of questions and complaints that arise, explaining that they can still use JSON, that you don’t have to rewrite the project in Erlang, and that yes it’s weird that Dell used to own RabbitMQ. And VMWare. And SpringSource. And that VMWare still owns it. Yes, it’s literally a list of companies you wouldn’t expect… can we move on?

Satisfied with how you navigated that, two weeks go by and you’re in a design meeting and someone pipes up asking why RabbitMQ is now part of the design diagram. Your ears prick and you jump in “Hey Frank, did you not see my Slack announcement a couple of weeks ago?” As soon as you say it, you realize that no, he did not — for he was on vacation. You quickly acknowledge it and say you’ll send it to him, and you search for it because you forgot to pin it in the channel, but that’s fixed now.

Two more weeks go by and you come across a defect ticket being worked, on the old, homegrown asynchronous message service. “How did this even make it into the backlog??” You look at the names on the ticket and DM them “hey guys, there’s some kind of disconnect, because I see this ticket, but we were supposed to stop using the old messaging service.” It’s news to both of them. They remember seeing the Slack announcement, but didn’t realize they should start using Rabbit right away. They thought it was just for new development. “Hmmm… I guess the announcement could’ve been clearer”, you realize.

Two more weeks go by and you’re looking over a giant pull request from Stan when you notice … no … it can’t be. He’s added a whole mechanism for pushing messages over WebSockets! You look at it again, and it’s true. It’s exactly what that cowboy is doing. He just completely ignored the announcement and spent weeks working on this giant component that replicates exactly what Rabbit is supposed to do for us, and fragmented the architecture in the process.

"What Slack announcement is this?"
<you send him the link>
"Oh. Sorry, I don't remember seeing this. I don't normally pay attention to Slack unless someone DMs me."
"Seriously?"
"Yeah, can't concentrate with it dinging me all the time, so I turned notifications off."

After you’re done slamming your head against the desk, he tells you that he actually considered RabbitMQ on his own, before dismissing it because he already needed the WebSocket connection for bidirectional communication and it would’ve made the whole message path more brittle to go over two connections. All of which are good arguments, and fits into the “good reason not to” clause.

The story has a happy ending because I’m not GRRM, but it easily could have not. And a fourth one might not. The point here is that there are all kinds of opportunities and good reasons for people to not internalize things. Here are some:

  1. They didn’t get the memo, for various legitimate reasons ranging from being out that day to technical glitches
  2. They didn’t understand all or part of it
  3. They skimmed it, didn’t think it affected them, and promptly dismissed it
  4. They don’t comprehend things well in that medium: some people need to see pictures and read things, some people need to hear them, some people don’t read Slack but do read email, some are the opposite
  5. They heard it, understood it, and forgot it anyway

As a leader, I feel like it is absolutely part of the job to be the positive version of Marty McFly’s 2015 boss, that tells him he’s fired via TV and three different faxes, for some reason. Send that Slack announcement, but also send it in an email, and say it out loud in three different meetings with slightly different people, and iMessage that one guy that doesn’t pay attention to anything but iMessage, and paste it in a document and file it in a good place in your document hierarchy and make sure it uses good, searchable words, and add a joke or a pun or something to make it memorable. And repeat it every time the subject gets anywhere close to it, until people audibly start groaning in your direction. That’s when you’ve succeeded.

In other words, for the important stuff, do whatever is necessary to make sure it gets through to the right people. Go to them. Meet them on their own terms, in whatever way they process information. Write everything five times. People are different, both from themselves and from machines. Embrace those differences, because it’s what makes a team a powerhouse of creativity. So you have to repeat yourself — it’s well worth it.

The Best Testers Are Scientists

It doesn’t take long to appreciate a great software tester. And it doesn’t matter if she’s a manual tester or writes automated tests, because what really matters are the types of tests being run: curious tests. Tests that don’t just discover a bug and quickly document it away in a ticket, along with the state of the whole world at the time of discovery. But instead, tests that try to find the exact circumstances in which the bug occurs.

The more defined those circumstances, the more helpful the ticket is to the developer and, ideally, will take their mind right away to the exact function that is responsible for the bug. In those cases, you can almost see the light bulb go off:

Tester: I’ve only seen the bug on the audio configuration screen, and it usually crashes the app after single-clicking the “source” input, but I’ve seen it a couple of times from the “save” button too. And it seems to only happen after a fresh install on Android 10.

Dev: ohhhh! That’s because the way we handle configuration in Android 10 changed and the file the audio source is saved in doesn’t exist anymore!

This is exactly the kind of dev reaction you want to a bug report. It’s an immediate diagnosis of the problem, which was only made possible by a very well-researched and described bug. But notice how that description could, with changes only to the jargon, have been written by an entomologist:

Entomologist: I’ve only seen the bug on a tiny island off the coast of Madagascar, and it’s usually blue with green spots, but I’ve seen a couple of them with yellow spots too. And it seems to only come out right after sunset in the wet season.

Which is kind of obvious when you think about it, because what do scientists do? They test the software that is our reality. Galileo’s gravity experiments is one of the more famous in history (and likely didn’t happen), but what is it, in software terms? He wanted to know if the rules of our universe took weight into account when pulling things toward the Earth. A previous power user, Aristotle, figured that the heavier a thing was, the faster it would fall. But that user failed to actually do any testing. So thank God that talented testers, like John Philoponus and Simon Stevin, came along and figured out that things mostly fall at the same rate through air, and then bothered to update the documentation.

What Aristotle did was assume the software worked in a certain way. Granted that he didn’t have the requirements to reference, but he probably noticed that you have to kick a heavy ball harder to go the same distance as a lighter ball, and he figured that the Earth kicks all things equally hard. That’s the equivalent of our tester above seeing the “source” input work on Android 9 and not bothering to test it on 10. Or seeing that it worked on the video configuration screen and not bothering to test it on the audio one too.

And that’s okay, because Aristotle was not a tester. He was more like a fanboy blogger. But what testers should be, is bonafide scientists, like Simon Stevin, who follow the scientific method:

  1. Ask a question
  2. Form a hypothesis
  3. Make a prediction, based on your hypothesis
  4. Run a test
  5. Analyze the results

In our example with the “source” input, after the tester saw it the first time, she probably did something like this:

  1. “why did it crash?”
  2. “maybe it was because I pressed the ‘source’ input”
  3. “if so, that’ll make it crash again”
  4. Relaunched the app, tried it, it crashed again.
  5. “okay, that was definitely the reason”

Aristotle might stop there and file the bug: “app crashes when using the ‘source’ input”. And the developer would try replicating it on their Android 9 phone and kick the ticket back with “couldn’t replicate”, and that whole cycle would be a waste of time. But our tester asked another question:

  1. “does it crash on this other phone?”
  2. “if it doesn’t, it’s a more nuanced bug”
  3. “I think it’ll crash though”
  4. Tried it on the other phone: it didn’t crash
  5. “what’s different about this phone?”

And she continued the scientific process like that, asking more and more pertinent questions, until the environment that our bug exists in was fully described. Which is exactly what you want in a bug report, because anything less will, in aggregate, be a productivity weevil, wasting both developer and tester time with double replication efforts and conjectures about the tester’s environment and back and forths. A clear, complete bug report does wonders for productivity.

So then, why not just teach all your testers the scientific method? Because it doesn’t work in the real world. We all learn the scientific method, but few of us become scientists. And I imagine that, just like in any profession, a not-insignificant number of scientists aren’t good scientists. Knowing things like the scientific method is necessary, but not sufficient to make a good scientist. You also need creativity, in order to ask the interesting questions, and more importantly, curiosity to keep the process going until it’s natural conclusion — to uncover the whole plot.

Tangentially, curiosity is a hugely important trait in great developers, too. But for testers, even more so.

My Slack Tips, 2021 edition

(There’s an updated version of this: My Slack Tips, 2023 Edition)

You know those recipe websites filled mostly with the backstory of how they discovered this amazing cookie recipe that changes their lives, and what joy it brings their three free-spirited, yet precocious children every time they make the cookies? This is kind of like that, so scroll down to “The How” if you don’t care about “The Why”.

The Why

I love Cal Newport‘s ideas about deep work. He writes a lot about how to protect your time so that you focus for stretches measured in hours instead of minutes, with no distraction. For Joel Spolsky the same concept was “getting in the zone”:

We all know that knowledge workers work best by getting into “flow”, also known as being “in the zone”, where they are fully concentrated on their work and fully tuned out of their environment. They lose track of time and produce great stuff through absolute concentration. This is when they get all of their productive work done. Writers, programmers, scientists, and even basketball players will tell you about being in the zone.

from “Where do These People Get Their (Unoriginal) Ideas?

Joel followed that up by explaining how task switching is bad, Jeff Atwood echoed him, Rands showed us how he kept distractions at bay and years later, after Slack arrived on the scene making this problem worse, how he used Slack. It being such an important topic — especially with the boom of remote work making Slack even more common — this is my version of that Rands article, with a few years of more Slacking under the belt.

For programmers, standard Slack policy should pretty simple, flowing out of the principle of guarding “the zone” fiercely:

  1. Turn off all notifications, except for when you’re being specifically @ mentioned
  2. Specific mentions aside, only read Slack during your downtime — between meetings or while your VM is rebooting or whatever.
  3. Don’t feel like you have to read everything in all the channels. Start with your favorite channels, and go as far as your downtime will let you.

For engineering leaders though, especially manager- and architect-types, it’s not so simple. The broader your scope is in the organization, the more pertinent info you will learn via Slack discussions, and the more value there is in being in a lot of different channels, so you can glean stuff like team Aardvark being in trouble because they’ve been complaining about their infrastructure daily for a week; or you can quickly unblock Bob who can’t remember where the functional spec is for the login screen, or focus a search API design discussion between Carol and Dave, and then build rapport with the other foodies in #random-food.

At the bottom of it all, underneath all of the turtles, programming is about communication between people. The source code isn’t for the processor, but for other programmers. And every successful team is a well-oiled communication machine. The best teams are the ones that have shorthand and can finish each others sentences. You just know that a team like that — and full of smart people — is going to be off the charts with productivity.

So as an engineering leader, possibly the single most important job we have is to do everything in our power to facilitate effective communication. Which these days, includes being a Slack Jedi. But, we also need to get some actual tasks done, so we can’t let it take over the whole day (at least not every day), which means there’s a balance to be found. Below, is how I found that balance.

The How

I’ll caveat these 11 weird tricks with the fact that this is what works for me, in my situations so far, and while your mileage will probably vary, at least some of this might be helpful. So:

  1. Tweak the in-app Slack notifications. The default notification settings inside Slack are mostly fine: it notifies you for DMs, mentions, and keywords. But I do make a few small changes:
    • I usually add a few keywords, like my name, team names, or project codenames
    • I go into the notification settings of any “system down” or other emergency situation kind of channels and set them to get notified of all messages in those channels.
    • In the macOS desktop Slack app specifically, in the “Notifications” section:
      • I uncheck the “Bounce Slack’s icon…” checkbox
      • If I’m in multiple workspaces, I also uncheck the “Show a badge…to indicate to activity” checkbox. Which keeps the badge icon from getting a red badge everytime someone says anything in any other workspace.
  2. Turn off all notification types from the OS, except for the badge icon. Not inside the app, but in the Notification settings of macOS/iOS/Windows/whatever, I have the Slack app set to not be allowed to make a sound, to never pop anything up, to never bounce anything — to never do anything at all, for any reason, other than (on Apple OSes) make the badge counter appear over the app icon. That way, I can get in the zone when I need to, and won’t get distracted by Slack, but can still glance at it, once in a while, at opportune moments to make sure it doesn’t have 57 urgent messages for me.
  3. Low barrier to enter channels. I’ll enter most channels as I come across them, because in the bad case, I don’t read them and in the worst case, I leave.
    • Slackbot helpfully and periodically reminds you about channels you might wanna leave, also.
  4. Low barrier to create channels. Whenever a topic starts getting enough airtime, or when there’s any other reason for it to have a dedicated channel, I create that channel. The more the merrier.
    • You can go overboard, but as long as the reason for the channel is clear, go ahead and create it. You can always archive it later if it was a mistake.
    • Channels are free and an organized Slack is a happy Slack, because it is way easier to find that important conversation about search API design if you know it happened in #dev-search.
    • If you actively want to foster conversation in a channel, consider making it private. People feel safer talking in them, and the Slack stats I’ve seen bear out the fact that the large majority of messages happen in private channels or DMs.
  5. Organize the channels in groups and make them disappear. This is a fairly new feature, I think from late 2020, but you can create channel groups in the sidebar, order the groups however you want and for each group, show all the channels or just the unread ones. You can also order the channels within the group in different ways.
    • I order them by priority within each group and show just the unread ones.
    • My groups are about related topics, ordered by how important they are to my role: my team’s channels and DMs in the first group, various dev channels in the second, then management channels, field support, water cooler, etc
  6. Embrace speed reading. The human brain is great at patterns, and you can quickly pick up the importance of a conversation from some key signals like what channel it’s in, who the participants are, how long the conversation goes for, key words being repeated, emojis being used, etc.
    • In some channels, like ones about nascent projects or downtimes, I read every word carefully — but that’s very much the minority.
    • For the majority, I scroll through at a good speed, slowing down only when my spidey sense tells me I’m speeding through something worthwhile. It works well — false negatives are rare.
  7. Embrace emojis. Not only do they make conversations fun, but they also convey tone (which is super important) and, as reactions to messages, they serve as pithy replies that add value to the conversation without also adding volume to it.
    • e.g., using a check mark to indicate that you read something without indicating judgement, or a dart that their message was right on the money, or an up arrow as an upvote — they’re small, very helpful gestures that significantly improve the conversation.
  8. Use reminders. When I do go into Slack, I like to at least clear all my mentions that turn channels red. But sometimes I can’t actually service a message. Maybe that’s because it’s asking a question I have to research, and I’m in the middle of something. Or I want to make sure that I follow up on a conversation after my current meeting. Or I got an action item in that meeting that I need to do tomorrow morning. I use reminders for all of that.
    • Most of my reminders are set on existing messages that I need to do something about
    • Some of them, I create with the /remind command
    • All of them, I snooze extensively until the time is right to work on them
  9. Don’t use threads. It’s quite possibly the worst feature in Slack. Some people love them, but don’t be those people. Terrible UX aside, all threads are is a way to hide conversations for no good reason. If you want to read every word in a channel, and you’ve read them all, but then Ethan decides to start a thread off of someone’s message 50 messages up, guess what? You’ll never even know.
    • Some people use threads because they came too late upon a conversation that’s scrolled away, and want to continue it; there is a better way: share the message into the same channel, and that posts it at the bottom, with your comment attached.
    • “But I don’t want to bother everyone in the channel with my thread”. The conversation either belongs in the channel, or it doesn’t. If it doesn’t, create another channel or group DM. But there’s no reason to hide it.
  10. Nudge people toward the proper channels. Most people are trying to get their job done and aren’t so worried about keeping Slack organized, so when they start a conversation about lunch in #dev-breakfast, it’s probably because they didn’t even know #dev-lunch existed, or because they had been talking about brunch and it morphed into lunch.
    • The point being that, as is generally the case, people don’t intend to break the rules and usually just make mistakes.
    • I, as Slack Jedi, have to minimize the chance of mistakes by establishing good channel-naming conventions and keeping topics narrow, but when mistakes do happen, it’s not the end of the world, and an innocent mention that the lunch channel exists is normally the only thing that needs to be done.
    • Be super nice and soft about it, because creating anxiety about the proper channels is kind of antithetical, since it then stifles communication.
  11. Try to mark everything read by the end of the day. I don’t always clear my email inbox every day (or longer), but I almost always clear my channels, to give me peace of mind.
    • On days filled with meetings, it’s harder, but I keep up with the @ mentions and high priority channels as I can, and then quickly scroll through the less important ones, just to make sure I’m not missing anything important.
    • Yes, the Slack FOMO is strong with me, but the clearing doesn’t take long. I’m in dozens if not hundreds of channels, in a Slack with hundreds of people and — active conversations aside — I can clear a day’s worth of chatter in about a half hour, knowing then that I’m on top of things and won’t be surprised the next morning by Fiona with a “so what are we gonna do about that nasty database locking issue?”

So that’s how I stopped worrying and learned to love the Slack: by putting it in its corner and giving it attention on my terms — no more and no less attention than is beneficial.

Be The Lorax

I am the Lorax. I speak for the trees.

“The Lorax”, 1971

This was Dr. Seuss’ favorite of his books. If you haven’t come across it, it’s a great fable about a woodland creature who keeps warning an industrialist to stop cutting down all the Truffula trees; but the guy doesn’t listen and proceeds to destroy the ecosystem.

The Lorax book cover

From time to time I feel like the Lorax, but instead of the trees, I speak for the developers, who are every software company’s most precious resource1. And I think this is a crucial, but often overlooked part of the software manager’s job: litigation.

At times the plaintiff, other times the defendant, but usually in opposition to someone named Taylor from another department, like finance or HR, who doesn’t understand engineers. Sometimes Taylor is someone very high up, who used to be an engineer a lifetime ago, but they’ve forgotten what it’s like, now that their days are filled with meetings about sales projections and synergy. And here you come, Sr. Cat Herder, with a request to change the new dress code policy.

“You might not have heard, but we had a bit of a problem in Marketing, so we had to institute the dress code. We didn’t want to, but you know Legal.”

You see, Taylor’s not a villain. Almost nobody is. They’re just trying to do their best, and one thing people outside of Engineering are pretty bad at doing is putting themselves in an engineer’s shoes. It would be easier for most people to pretend they were a parakeet. But this is where you come in — you who have knowledge of the way of life in the dark cave of Engineering.

So you use your nerdy charm and wit to show that having a policy is fine, but that it just needs to be tweaked, because while the developers have no intention of causing problems, roughly half of them will quit before dressing “professionally” at work. And a third of those, have not worn long pants or closed toed shoes in years — and won’t start now. Not when they can get a good job by just whispering the words “I’m looking for a change” into the ether that’s continuously monitored by eager recruiters. Because as it turns out, those developers are a lot of the best ones we have, and they’re past putting up with well-intentioned policies. And if even one of them leaves, it’ll cost us a ton in recruiting, on-boarding, shipping delay, and general business risk, which is surely worth tweaking the policy to allow cargo shorts and sandals.

“But it’s 45°F outside!”, Taylor exclaims.
“It doesn’t matter, because they’re almost never outside,” you reply, and then continue with a whisper, “and they’re really stubborn.”

If you do this well, the policy will get tweaked before the developers are even told what was in the policy email they deleted, and the builds go on without a blip.

Sometimes you have to be the change you want to see in the company, which is a bit harder. Convincing the powers that be to allow flexible hours, or to get Engineering more expensive laptops, or that the savings were not worth switching to Microsoft Teams — these are the kinds of things that might need a Powerpoint. A beautiful, well-researched, entertaining Powerpoint, which will take a lot of your precious little coding time.

But these are the things you have to do as a good manager and the sacrifices you have to make, because you are the Lorax, and you speak for the devs. Though… hopefully better than the Lorax, in that you actually succeed in preventing the collapse of the ecosystem.

Footnote

  1. Just to be clear: I was using “resource” metaphorically. As much as nature abhors a vacuum, I abhor calling people resources. Trees and laptops and StackOverflow articles are resources — people are not.

Software & Opera, Part 3: Design & Test

This is the third and final part of a series on opera and software. Part 1 explained why they’re related, part 2 explained the development process used to produce an opera, and this part explores what can be applied to the software development process.

But first, the disclaimers: I’ve never worked backstage anywhere else, so I don’t know how other opera houses operate — they could be exactly the same, or wildly different. Also, everything in this article comes from my experience, and nothing is from Sarasota Opera — they don’t know I’m writing this (and hopefully, they’ll be pleased if they do ever come across it). Finally, what I do know of the production process comes from years of observation from the sidelines, from the role of a super, so some stuff in here could be just plain wrong; I’ll try to make it a point to underline where I know my knowledge is shaky, but there might be things I’m more sure about than I should be.

Let’s just dive in by recapping the operatic development process using software terms:

  1. The product manager writes functional specs (the libretto)
  2. The architect (composer) writes technical specs (the sheet music) based on the functional ones
  3. Management chooses a reasonable release date
  4. About a year before the release date, development begins on the infrastructure and wireframes (stage, sets, costumes), congruent to the functional and technical specs.
    • The infrastructure may well be sourced from elsewhere, or built in-house — whatever makes the most financial and artistic sense
  5. The main components (principal singers and stage management) are identified
  6. With roughly six months to go, the technical lead (conductor) learns the technical specs (sheet music) backwards and forwards and begins work on the main components (the principal singers)
    • The same as how, in software, the architect and the lead developer could be the same person so in opera, the composer and conductor could be the same person: Verdi conducted the first runs of all of his operas, and so have most composers. But after that first run, their operas are conducted just as well by other conductors.
  7. The UX designer (director) begins fleshing out the UI (choreography)
  8. Development begins on smaller components (auditions for smaller parts, chorus)
  9. Support infrastructure is stood up (props, rehearsal rooms, living arrangements)
  10. Management meticulously schedules acceptance testing (rehearsals), including bug fixing time.
  11. A couple of months before release, major development is done (the choreography, principal parts, sets, and costumes are all largely in order), the infrastructure is in place, and developer testing begins (all cast members rehearse at home)
  12. A month before release, developer testing is complete, and development is largely frozen: the infrastructure is ready, the entire cast is on site, and know their parts and music. Developer testing on the main components has been especially rigorous.
  13. Acceptance testing begins:
    1. The testing phase is the climax of the process. All hands are on deck, for very full days, for the duration.
    2. It takes place in a lightweight mock environment (the off-stage rehearsal room) with mock props and costumes
    3. The mock environment is designed to be easily reset and changed from one test to another, which is not the case for the production environment (the stage with sets)
    4. The architect, UX designer, developers, and management are all present
    5. Each workflow (scene) is thoroughly tested
    6. The testing harness supports starting from any point in a workflow, stopping at any point, and resetting to any point, so that the entire workflow doesn’t have to be run every time, making it possible to easily test every individual feature
    7. The testing harness supports various levels of integration testing, from interaction of only main components with mocked minor components, to full end-to-end testing
    8. Project management keeps track of each test’s outcome against the specs and design
    9. Project management files and tracks enhancement requests and bug reports (changes and corrections to choreography and singing) made by the developers and designers, who update specs and designs as needed
    10. The tech lead and UX designer make tweaks as needed and file bug reports, which are worked on with high urgency
    11. Once the tests are going acceptably well, testing moves from the mock environment and into a real one: the actual stage
    12. More enhancements and fixes are made until the UX designer and developer are happy with the outcome
    13. Beta users (coworkers, friends, early adopters) are brought in for final testing. There are very few, if any, surprises at this point.
  14. The product is released on opening night
  15. A point release or two may quickly follow, with minor changes (in the first couple of performances)

That should all be very familiar, but how does that differ from the process in most software shops? From my experience, it’s the beginning and the end: the design and the testing. Most shops focus on the middle: the development itself, with testing being a necessary evil, and design often being a quick drawing with some boxes and arrows, maybe just in someone’s head.

Don’t Focus on the Coding

The above process takes probably about 18 months, and the first half of it is all design. Main feature development takes about six months, and testing takes half of that again. So a full two-thirds of the process is design and testing — not development. On top of that, at the end of that six months of development, everything is already well-tested by the developers, with unit tests and rudimentary integration and functional tests. Which means that even less of that six months is actually spent on developing features; maybe as much as half that time. Counting that developer testing, we may be looking at nine months of design, three months of development, and six months of testing.

Design

At this point, you may be aghast at the idea of spending less than 20% of a release cycle coding, and 50% of it designing. The important thing to remember is that designing is programming. The difference is that it’s at a higher level, and it consumes almost all of the research time. It’s a more efficient, and more disciplined, approach than starting to code with only a vague plan, and changing direction as problems arise, pausing to learn new technologies, finding the right widget library, and then realizing that the protocol you started with doesn’t support one of your main features as you wrote it, which means going back to the drawing board.

That going back and re-coding, that’s what wastes a lot of time. That’s why the opera has so much design up-front and why a release date isn’t even chosen until the detailed design is complete, and thoroughly understood. And the opera is not alone, because every disciplined engineering profession does the same. I’ve never worked in civil engineering, but I’m pretty sure that every iron nail in a highway overpass is designed and placed before one worker arrives on site. A big design up front — not coming down a waterfall, but rather in a process that is flexible and agile — documented in great functional specs, is the hallmark of the timely and efficient building of anything.

When done well and thoroughly, it front-loads the vast majority of the variability. If you spend the first half of the cycle coming up with a great design, the last half should be an exercise in typing. The great screenwriter Aaron Sorkin said “You have to think about what you’re going to write before you write it”. Software works exactly the same way. The more specific and detailed your thinking, and the design document that comes out of that, the less surprises there will be in the development and testing phases.

This means the design should address gotchas head on: every technology used should be well-understood, it should be vetted to make sure it works like the architect thinks, and the design document should be as detailed as possible, with APIs, flowcharts, state machines and anything else that can reduce risk and variability in the second half of the cycle. Plus, that document should be very readable, and even entertaining, so that the variety of audiences that read it (devs, managers, QA, tech writers, etc) don’t fall asleep doing so.

If I’ve taken one thing away from my opera experience, is that it’s the important of a thorough design up front, in which literally every note is written before a singer is hired.

Testing

If there are two things I can take away from the opera, the second is definitely the outsized scope of testing. At the opera, testing is not only built into every part of the process — from design to development to release — but everyone is also laser-focused on it. Conductors play their music on the piano to make sure it doesn’t just sound good in their head. Singers practice for hours a day with coaches, who you can think of as the developers of singing. They essentially run unit tests all day long, making changes until the output (the voice) matches the spec that is the sheet music.

And once rehearsals start, everything falls in place, for the most part, with ease. There are always changes because a singer turned out to be taller than the director imagined, so he has to be placed further downstage; and three people in the chorus constantly put the emphasis on the wrong syllable of some Italian word they’ve never heard of, so the maestro has to explain Italian pronunciation yet again.

Enhancements and bugs are built into the acceptance testing phase, but they’re assumed to be minor. Easily changed and corrected, with relatively little effort from the developer. You don’t find out that the soprano can’t hit the high note at this stage, because she auditioned long before. You don’t find out that the set design is all wrong, because an intricate miniature replica was built first. If the main singer gets the flu, there’s a cover waiting to step in, having shadowed the singer at every rehearsal. Everything has been vetted, pre-tested, and every contingency addressed so that there are no show stoppers. Even without any changes at this phase, the opera would be good. The changes here are the little details that make it great: a flawless, amazing performance.

To get there, the testing has to happen in design and during development first. The acceptance testing has to involve all the roles in the production. And it has to be properly planned, with ample time scheduled for it, so that its success is the number one priority for everyone during that time.

Conclusion

By now, I’ve been to dozens of operas, many on opening nights, many in the audience and many backstage or on-stage, and I can count on one finger the number of major blunders I’ve experienced. It was about the fifth performance of a production of Turandot, when the king repeated the same line twice. People that weren’t paying attention to the Italian wouldn’t have even noticed, but someone else played the part the rest of the run.

I’ve seen many more small issues, like people dropping prop coins or papers, walking out the wrong exit, or forgetting to take a jacket off. But the lack of issues in general is astounding, to say nothing of the ability to release on time to the day, and on budget, year after year after year. All of that comes solely from the company’s ability to front-load risk, which then lets them plan the release with certainty, and devote ample time to testing and refactoring, so that the final product is an absolute gem.

Software & Opera, Part 2: The Operatic Development Process

This is the second of a three-part series on opera and software. Part 1 explained why they’re related, this explains the process used to produce an opera, and the last part explores what can be applied to the software development process.

Before I go any further, some disclaimers are in order: I’ve never worked backstage anywhere else, so I don’t know how other opera houses operate — they could be exactly the same, or wildly different. Also, everything in this article comes from my experience, and nothing is from Sarasota Opera — they don’t know I’m writing this (and hopefully, they’ll be pleased if they do ever come across it). Finally, what I do know of the production process comes from years of observation from the sidelines, from the role of a super, so some stuff in here could be just plain wrong; I’ll try to make it a point to underline where I know my knowledge is shaky, but there might be things I’m more sure about than I should be.

Pages from an 1859 libretto for Ernani

The stuff that happens early on is what I’m least sure about, because I generally start working on an opera during off-stage rehearsals, which is essentially the “acceptance testing” phase. But from what I’ve gathered, this is what happens before then:

  1. The libretto is written. This is like a movie script and contains the dialogue and stage direction. For most operas performed today, this was written long before your grandparents were born.
  2. The libretto is set to music. The composer takes the dialogue (and the gaps therein) and creates music that sounds nice to people who like operas. Again, this has usually been done at some point in the 1800s, but even for new operas, it would be at least somewhat before anything else happens. And now you have an opera — at least, on paper.
  3. The opera is slotted into a pre-existing Friday opening, about 18 months in the future. By the time one season starts, the next season’s roster of operas and their exact release dates have already been made public.
  4. Set and costume design are started. This is one of the longest phases, because a set may have to be created from scratch, and sets are big and can get complex. They actually take up most of the backstage area. Renting them is an option, if the size and shape of the stage is pretty standard. Costumes also may have to be made — sewn in the opera’s costume shop — but sourcing ready-made ones is time consuming too, since an opera with a full chorus and a bunch of settings could easily require over 50 costumes. And sizing is a thing to deal with on top of it all.
  5. About a year before opening, the principal singers and stage management are hired and contracted for the work, which actually starts some months later. There are usually about six principal singers and maybe a couple of more minor roles.
  6. The conductor starts learning the music and hires the orchestra.
  7. The director starts choreography design, figuring out what happens in each scene: where, when, and how people enter; what path they walk; how they act, what they physically do; and where, when, and how they exit the stage. Some of this is dictated by the libretto, but most is not.
  8. A couple of months later, general auditions take place for the chorus, which is about two dozen people.
  9. A few months before opening, props start to be created or sourced: swords, treasure chests, torches, faux violins. The set and costumes are done, but not tailored.
  10. The rehearsal schedule is completed — for two full months of rehearsals. Since four operas are being produced at the same time, only parts of each opera are rehearsed in each session (usually one particular act or scene), and rooms and the stage have to be scheduled so that rehearsals can happen in parallel. Also, the chorus usually appears in all four operas, so the scheduling has to take into account that they can’t be in two places at once.
  11. A couple of months before, the cast is all set and the various design work is mostly done. The cast starts rehearsing the music on their own.
  12. About one month before, the cast arrives on site (they come from all over the country, and internationally) and rehearsals actually begin.

This is the point where I enter. By the time of my first rehearsal, the principal actors have been rehearsing for weeks, and the larger chorus for some days. Principals have something like 10x the stage time as the chorus, so they have a lot more to work on.

The rehearsals early on are held in a large rehearsal room instead of the stage, to minimize labor costs associated with setting the stage. Instead, the floor of the room is marked with tape outlines of the sets, with different colors indicating different scenes. So Act 1 might be in red tape and outline some stairs and a couch, while Act 2 might be in blue tape and outline the same stairs, plus two doors and a table with chairs. When you rehearse Act 2, you walk around the blue tape and ignore the red tape.

There are generally a lot of new people, so everyone gets a name tag in the beginning. In each rehearsal, in addition to the performers, the conductor is present, along with a pianist, the director, a stage manager, and two assistant stage managers: one for stage left, one for stage right. Principals all get cover singers, to put the bus factor at 2, and their covers are also there.

Since the choreography is already designed by now, the director tells everyone what’s going to happen in the scene. No one is in costume, except principals might wear something a drunk person might think is their costume, to get used to walking around and singing with, say with a sword or a heavy coat. The singers have already memorized their music, and performed it in front of a maestro and gotten notes on what to change.

An important thing to understand is that everything in an opera is cued by music, and the music is cued by the maestro. She moves her baton at a certain tempo, and all of the singers and orchestra musicians keep that tempo. When she stops, they stop and when she starts, they start. If they start to lead or lag her, she stops and yells at them. “Can everyone see me? Can you, Jeremy? Because if you can, why are you half a measure behind?” The conductor is essentially the system clock, for a multi-processor system. Or maybe an NTP server.

Because of this, the cast’s actions are also cued by the music and it follows that the director’s directions are usually given relative to some piece of music: “when Isabella gasps, you three move upstage, and you go put your hand on her shoulder. Then, on the beat after ‘da dada da dun’ everyone but the Don exits the stage. Ok, let’s take it from two bars past 62”. It’s amazing to me that the singers can start from any part of the entire opera, with a second’s notice.

Stage management keeps a giant binder with the sheet music for every role and marks where in it things are supposed to happen — like stage entrances and exits. After everyone verbally understands what’s supposed to happen, we run that part of the scene, with piano music. Here usually, someone omits doing something — walking somewhere, facing someone, fake laughing — something got forgotten or misunderstood in the space from the director’s mouth to the cast, so we reset and run again. Sometimes, the director, having seen it run with actual people, decides to change where people stand or face or who they fake banter with, so we reset and run again. Later, when we rehearse on stage with the actual set, sometimes things are tweaked even more. And if the singing is off, the maestro stops the music, yells admonishments, we reset and run again.

Once the director and maestro are happy with the way things are running, they layer in any additional people in another rehearsal. For example, sometimes we have rehearsals just with the supers, or the supers and chorus but not principals, for various scheduling reasons. And when everyone knows their parts well enough, they’re all brought in to the same rehearsals.

The off-stage rehearsals are generally splayed over three weeks, with each scene being rehearsed about three times, depending on the complexity. Once in a rare while, things might be going so poorly that they might add rehearsals; as an unpaid volunteer doing this for 20 hours a week on top of your day job, that’s what you really don’t want.

Finally, it’s about three weeks before opening, and we get to rehearse on-stage! There are only a handful of rehearsals left at this point, but most of them are of the entire opera, start to finish:

  1. A couple of rehearsals with only the piano accompaniment still
  2. A piano dress rehearsal — meaning still only the piano, but full costumes for everyone, though only principals get makeup
  3. A rehearsal with the orchestra but no costumes
  4. Finally two full dress rehearsals with the orchestra.
    • These last two are run exactly as if they were live performances, with beta testing users being brought into the audience, in the form of friends, family, and big shot donors.

During these on-stage rehearsals, the lighting people are also testing the colors and intensities to make sure they create the proper environment and mood, the costume and makeup designers check to make sure everything looks good from the audience, and the director walks around the whole opera house to make sure various vantage points can all clearly see the important action.

For the last rehearsal, they generally bring in a couple of classes of middle or high school students, and again: they’re run exactly as they will be on opening night, with PA announcements, supertitles, intermissions, curtains, etc. — even the bows at the end are rehearsed. The only differences with respect to a real performance is that the conductor might still stop the music if things are going too wrongly, she gives notes to the orchestra at the beginning of each act, and the director hovers around the whole house looking for final tweaks to make.

After opening night, things are mostly on rails. The director might issue notes with small things the cast is doing wrong or she’d like to change, someone might need musical coaching if they’re too loud or soft or whatever, but by and large everything is set at that point, and it all just manifests like a clockwork for a dozen performances. That is actually the inflection point for when it gets really boring. After the jitters of opening night, it all becomes so routine that there’s not much excitement in it anymore — but that’s good, because then you can finally focus on the performance.

In the next and final part, we’re going to talk about how all of this applies to software development.

Next: Software & Opera, 33 1/3

Software & Opera, Part 1: The Music, Not The Browser

Over the past decade, I’ve done eight seasons with the Sarasota Opera. The usual reaction I get is either “Wow, I didn’t know you could sing!” or “I could totally see you as an opera singer!” I’m never quite sure if that second one is meant to be a compliment, but in either case: I can’t hold on to a note any better than to a cat. So instead of a Pavarotti-type, I’m what’s called a supernumerary, which is Latin for “extra numbers” — a non-singing extra, normally just shortened to “super”. (Because I’m male, they usually refer to me as a “super man”, which is awesome.) Also, Sarasota Opera is a very professional house — which is actually the reason for this essay — and only hires phenomenally qualified singers, who’ve formally trained for years. I, on the other hand, still haven’t figured out what a “clef” really does.

My parts are usually some kind of guard or soldier off in the background, my only reason for existence being to underline the importance of other people by escorting them on and off stage — or because they’re prisoners. Sometimes the parts are beefier, like a very cool executioner in the degradation scene in Jérusalem, or a nimble thief that scales the wall to kidnap Gilda in Rigoletto. My last role was a guy in a band in La Wally, faux-playing a faux-violin on top of a table, while trying to keep time with the actual violins in the orchestra pit, during a lively party that, of course, ends in disaster.

La Scala of Milan

At this point, you might be thinking “this is neat and all, but what does it have to do with software?” — and that’s a perfectly valid thought. The answer is that over these long years, I’ve noticed some very interesting parallels between the production of an opera and the production of software.  From my first day there back in 2010, Sarasota Opera amazed me with its clockwork precision, streamlined efficiency, and amazing courtesy and professionalism toward everyone, from principal singers, to lowly supers. And every year but this one, they deliver five operas: all on-time (same Friday every year), all impeccably sung, directed, and designed, generally getting great reviews, and involving a crew of several dozen people over a timeline of more than a year.

Can the average software company make the same claim? Apple and Google can: iOS and Android are released on a yearly cadence, like clockwork, and for the most part, they are masterful works with serious bugs being rare. However, in my experience, most other software releases are late and buggy. They’re probably buggy because they’re late and were rushed out the door, and they’re late because things didn’t go according to plan. So how does the opera — in essence a small, non-profit business — stay on plan and deliver such consistent and high quality results? What parts of that are and aren’t applicable to software development? And how much does the operatic world even have in common with that of software?

To answer the last question first, a lot:

  1. Both follow the same process: design, development, testing, narrow release, wide release
  2. Both produce highly technical works of art. Music, orchestration, choreography, and stage production are all, of course, highly technical. And if you don’t think software is a work of art, just think about how important good design is to user adoption.
  3. Both have audiences that are intolerant of bugs
  4. Both are staffed by highly trained professionals
  5. Both have administrative staff managing “the talent“, plus support staff making everything actually work.
  6. Both have the same constraints: deadlines, budgets and headcounts
  7. Both require a great deal of planning and design in order to be done well
  8. Both have a number of specialized departments that work in concert (pun intended): developers, testers, DevOps, ProdOps, UX, and product management on one side vs singers, orchestra, props, costumes, makeup, lighting, set design, and stage hand, on the other.
  9. Both have hundreds of components interacting with each other, even in parallel. Opera is generally massively parallel, especially if counting the orchestra.
  10. Both are run from compiled code: musical notation is conceptually the same thing as binary code, with each singer or musician being the processor that executes the instructions for their particular role

There are more, but ten is a nice round number. And hopefully the reason this essay exists makes more sense now. If you’re still keen, the next part talks about how an opera is produced and finally in the third part, what can be applied to the software development process — which is that projects should focus on design and testing, rather than coding. Oh wait… did that just spoil the ending? Because if not, you should also know that the other two parts are much longer, too.

Next: Software & Opera, Part Deux

Two More Annoying iOS 11 Bugs

So far, iOS 11 is maybe the buggiest release Apple has put out: it’s had eight updates to fix a bunch of issues, like the infamous A ⍰ bug, and while you’d think they’d have ironed most serious things out by now (11.2.1), I ran into two more, and verified them with Apple support on the phone.

I’m posting them here because I scoured the Internet for answers on them and got nowhere, which is why I resorted talking to Apple on the actual phone, and hopefully this will help you avoid doing the same thing. I’ll also update this post if and when they get fixed.

Bug the First: Edit and Share buttons are grayed out on some photos

This one is straightforward and super annoying and mind boggling: I can’t edit or share some small minority of photos. There seems to be no rhyme or reason as to which photos. The guy I talked to Apple couldn’t find the issue in their database, even though there’s at least one thread on it on discussions.apple.com (though it’s been erroneously closed).

Workaround: import the photo into an app that’ll let you re-save it. I used Camera+.

Bug the Second: On the first iCloud backup, you can’t choose what to (not) back up

Let’s say you have 5GB of Photos and you wanna do an iCloud backup of just the phone settings, because the photos are backed up already in some other way, and you got a new phone that you want to transfer your settings to. The settings themselves should be around 200MB.

Assuming you haven’t wasted money on iCloud and your limit is still 5GB, even if you have nothing in iCloud, when you turn on Backup it tells you there’s not enough space in iCloud, because it’s trying to upload your photos also.

Image result for ios 11 this iphone cannot be backed up because there is not enough icloud storage available

That’s it: you’re stuck. There is no way to get past this without buying more iCloud space.

In previous versions of iOS, you could switch off Photos, for instance, as well as iMessage and everything else, so you didn’t end up uploading the entire phone’s contents to the cloud. And you can still do this, on a backup-by-backup basis, but only — and this is the maddening part — if you already have a backup in iCloud.

Otherwise, there’s no way to control what goes up there or even see what the Next Backup Size will be.

Image result for ios 11 choose what to backup

Choosing what to back up was so easy in iOS 10, and I was so incredulous that it wasn’t in iOS 11, that I went to the Apple Store so they could show me what pathway I was missing. After that went nowhere, I brought it up with the guy on the phone, and he confirmed this is how things are now.

There’s a StackExchange question out about this, but the problem seems to be too nuanced to have made it onto Apple’s radar.

Workaround: use an iTunes backup to restore the settings onto the new phone.

I’ll update this post if and when these things get fixed.

Home Theater Setup 2016

I love big screens and I cannot lie. My first major purchase after I got my first professional job was a projector. My first major purchase after I got my first house was a new projector. And I got an even newer projector late in 2015. With it came a refresh of the whole setup.

This is now my third projection system, and after a lot of research, I feel like it’s the first that’s actually set up really well. The first projector wasn’t 1080p and I didn’t even have as much of a screen as a really uniform white wall, so… not a great setup. The second generation was a big improvement, but I didn’t do any calculations and so some people thought the screen was too big to watch from the couch. Either due to finally applying the maths, or because the third time’s a charm, this version 3.0 is very much on point, and so this post lays out what I did in excruciating detail, because good documentation is the cornerstone of process improvement.

TL;DR: I put an Epson PowerLite Home Cinema 5030UB on an OmniMount wall shelf 11′ away from a 110″ Elite Screens SK110XHW-E24. There’s VIZIO S5451w-C2 soundbar behind the screen, a Monoprice 40′ active HDMI cable going through the ceiling from an outlet to the soundbar, and another from the soundbar to the projector. Into the outlet, I usually plug in an AppleTV 4, but sometimes I plug in my PC. And because the soundbar is behind the screen, there’s a BAFX IR Repeater with the IR receiver on the screen housing, and the emitter behind it.

Total equipment cost in 2016 dollars: 3250$

Absolutely _not_ my home theater system

The Screen

I had to start with the screen because the projector needed to go into a family room attached to an open kitchen, which limited me to two walls of roughly the same size. The room itself is 11′ square, but in the direction that goes into the kitchen, the projector can go back as far as my wife would let me, which was about 15′, so that “it’s not actually in the kitchen, you know?”.

However, the bigger concern was brightness. The room is not very well light-controlled, and I need to be able to watch certain things — like football games — in the daytime. (The further back a projector is, the less bright it will be on the screen). We also have a TV facing out toward the kitchen, so putting the projector on the other wall would limit me to 11′ projection distance, but allow me to have both TV and projector running at the same time, as opposed to the screen covering the TV when it’s down. And you know what that means: simultaneous viewing of both NBC and CBS election coverage this past year. Oooooh yeah.

With the wall picked out, I calculated the screen size to buy: 11′ is 132″, and THX recommends screen sizes be 0.84 of the viewing distance, which works out to 110″ screen. (If you subtract the length of the projector itself, it works out more to like 100″, but I like screens being a bit on “bigly” side.)

For vertical placement there are two general guidelines:

  1. A rule of thumb that the bottom third of the screen should be below the viewer’s eyes, which are generally at 38″ for a seated adult
  2. That the top of the screen should be no more than 15 degrees above the viewer

For a 110″ screen, which has a height of 54″, the one-third that’s supposed to be below the viewer’s eyes is 18″. That leaves 36″ above the viewer’s eyes, and at an 11′ viewing distance, the top of that screen would be 15.3 degrees above the viewer, so that works out perfectly.

Screens generally have a 2″ black border all around, so that makes the entire screen 58″ tall, and brings the one-third that’s below the eyes to 20″. If the eyes are at 38″, that means the screen’s bottom edge is 18″ off the floor, and the top edge is 76″ off the floor. Unfortunately for me, there was a 96″ tall door frame on part of that wall, so I had to get a screen that had an additional 24″ of black border at the top, so I could essentially hang the housing up that much higher than where the actual white part of the screen needed to be.

With that added requirement, basically the only option I had was the the Elite Screens SK110XHW-E24 for 460$, which was a 110″ screen with 1.1 gain and 24″ drop. I’m not sure if I’d know a bad screen if I saw one, aside from obvious flaws, but I definitely don’t mind this one, and it goes up and down like a champ.

The Projector

Once I had the screen picked out, I started looking at the projectors rated highly on projectorcentral.com. Three stood out:

  1. Sony VPL-HW40ES – 2500$, editor’s choice for 2014
    • According to its calculator, for a 110″ screen with gain of 1.1, placing it at 16′ away would mean using a 1.07x zoom and the image brightness would be 20fL; 13’9″ for 22fL, 11’10” for 24fL
  2. Panasonic PT-AE8000 – 1850$, editor’s choice for 2015
    • According to its calculator, for a 110″ screen with gain of 1.1, placing it at 14’10” away would mean using a 1.47x zoom and the image brightness would be 20fL; 13’9″ for 22fL, 13’1″ for 24fL
  3. Epson PowerLite Home Cinema 5030UB – 2300$, editor’s choice for 2013
    1. According to its calculator, for a 110″ screen with gain of 1.1, placing it at 16’4″ away would mean using a 1.41x zoom and the image brightness would be 20fL; 15′ for 22fL, 13’9″ for 24fL

After some internal debate, as you may have read above, I went with the Epson, for a few reasons:

  1. I liked the previous Epson PowerLite Home Cinema 6100 I had
  2. It had the most reviews of any of these on Amazon (139 reviews, 4.5 stars)
  3. Great zoom and brightness

I also looked at the 5030UBe, which has the interesting wireless HDMI feature, but apparently it has connectivity issues, especially with gaming, so it’s not worth the extra price tag if you can’t reliably use it.

The Speakers

One of the great problems with front projection systems — if not the greatest — is sound. The virtually universal advice is that you need to get a good receiver, run all sound through that, and just run an HDMI output from the receiver to the projector.

Well, I’m a minimalist, I only have two inputs for the big screen (Apple TV and my PC), I’m not an audiophile by any means (stock TV speakers sound great to me) and on a philosophical note, I absolutely will not buy yet another gadget to make up for the fact that projector manufacturers just can’t seem to bring themselves to make simple Audio Out ports be a thing on their devices.

Throw in the annoyance of running speaker wire, and when I saw a well-reviewed Vizio soundbar on sale, and it had wireless speakers, I bought it. It’s the 2014 model, so it was old when I got it, and not even available new anymore. But, I love it.

At first, in an effort to avoid a run of cabling, I tried to have HDMI only go to the projector and to get the audio to the soundbar via Bluetooth from my AppleTV and my PC. But while music worked fine, video absolutely did not: the last syllable to many words would get dropped out, and it was just unwatchable. So I regrouped and ran HDMI to the soundbar and then another HDMI cable from the soundbar to the projector. The soundbar picks off the audio and sends the video to the projector, which means I just have two HDMI cables running through the walls and ceiling and it works beautifully.

The subwoofer is wireless and hangs out hidden away in the TV stand. Two wires come out of the back of the stand and go behind the couch and up to the rear speakers.

The Accessories

HDMI

For the first leg, from outlet to soundbar, I needed a 40′ cable, which meant I needed an active one, that deals with attenuation over that long of a haul. Monoprice is where I usually get cables, and this was no exception: they have 40′ active HDMI cables for 35$ and after one year, I have zero complaints. But do note that they are unidirectional: you have to have the right end in the soundbar or projector. I got two of these, one for each leg.

Now, a word about installation: for my second generation system, I waited for a cool November day, ran the cables in the ceiling myself, and I have the scars to prove it. Literally: I sliced my knee on an exposed bracket in the attic to what seemed like the bone, and had to superglue it shut (to avoid going to the ER and having them do the same). For this third generation, I was going to repeat history. Then I realized that due to the position of the soundbar in the house, I couldn’t even figure out how to physically get a cable there, much less how to stay conscious for that long in the Florida heat, that in an attic, approaches Venus conditions. So I hired a professional home theater outfit and both cringed and thanked the seven gods while seeing what they went through. Worth every penny.

Infrared

Second, in an effort to make things look nice, I mounted the soundbar behind the screen. I already had to mount the screen housing about a foot off the wall, so that the screen would drop over a set of drapes, so I had clearance back there. And really, I had nowhere else to put the thing. To my great dismay though, on testing, I found out that with the screen down, the infrared from the soundbar remote control would only work if I walked over and pointed it behind the screen and up to the soundbar. Which really sucks to do when you’re trying to enjoy a movie but it gets loud enough to wake the baby — and the baby is literally the last person you want waking up when you’re trying to enjoy a movie.

To get around this, I bought a BAFX IR repeater for 28$. You stick a small IR receiver on the projector screen housing, plug that into a box that’s mounted and hidden behind the housing, and then plug in an IR emitter that points to the soundbar. Boom! You’re now a light bender.

This particular gadget is a little overkill for this purpose because it could control up to 8 devices and I just needed one, but I couldn’t find anything good for less money anyway.

The repeater is _not_ pretty

Future Improvements

So that’s it for now: pretty simple setup, great results, mostly thanks to the calculations and planning. But nothing’s ever perfect, so while I’m happy with version 3.0 of the Jiva Projection System, I’m definitely planning on upgrading to 3.1 and 3.2 when I “get the time”.

First, I’d really like to actually mount the rear speakers up on the wall someday. Right now, they’re just on the floor behind the couch. Honestly, this is because (again, I’m the opposite of an audiophile) I find them more distracting than anything. Whenever I hear a distinct “rear” sound, I don’t think “oh wow, how realistic”, but more “WTF was that noise?!”. Which means that a lot of the time, surround sound is just off. But, maybe it’s like beer and I just have to acquire a taste for it so I can hang with the cool kids.

Second, I really need to put a power outlet behind the projector. There’s an HDMI outlet there that the nice home theater people put in, but they weren’t licensed enough to run power behind the wall. So now there’s this power cord that just runs down the wall, that a toddler can and would definitely like to tug on. And yeah, I could just drill two holes in the drywall and run the power cable behind it, but that’s not up to fire code, and insurance is a bitch. So I’m currently trying to either work up to running high voltage wire myself, or getting an electrician to do it. And I really don’t want to do the latter, because while delegating is par for the course at work, actually doing stuff is what makes me wake up in the morning.

Third, I probably should get a fancy Harmony remote, because between the projector, soundbar, TV, cable box, Apple TV and homemade DVR, we literally have half a dozen remotes. But, 95% of the time, the Apple TV remote is the only one we use. And I just can’t bring myself to spend 150$ or more on something that’ll get used once a week and only very marginally improve my life. But, I’m keeping it on the backburner as a birthday wishlist item, for when I become the man who has everything, so that I don’t wake up to find myself entombed alive in a cemetery in Mexico.

Setting Up Ubuntu For Android Development, 2015 Edition

At work I use a MacBook Pro because that’s what they give us. At home, my desktop is Windows because of Windows Media Center and gaming, but I also recently rescued a Dell laptop that now has Ubuntu on it. While setting up that laptop for Android development, I ran into a couple of gotchas and found some shortcuts; so when I set up a new VM for a new project on my desktop, I decided to document the steps for reference.

To wit, at the end of the following procedure, we’ll have Android Studio running on Ubuntu 14.04.3 (Tired Trampoline) on VirtualBox 5.0.10. If you need a lighter VM, these instructions also work with Lubuntu.

  1. Download VirtualBox (for this guide, 5.0.10), if you don’t have it
  2. Set up a new VM
  3. Download the Ubuntu Desktop ISO (for this guide, 14.04.03)
  4. Load it into the VM’s optical drive, boot up the VM, and start the install
  5. After answering the setup questions in the beginning, go watch TV for a while
  6. When it’s done, eject the install ISO from the VM and reboot
  7. It’ll boot up in glorious 640×480 mode; insert the Guest Additions CD image, to do stuff like allow you to use a decent desktop resolution:
    VirtualBox Guest additions disc

    • A dialog will come up allowing you to automatically run the script on the disk, but if something goes awry, open a terminal, go to the disk and manually run the script:
      cd /media/{username}/VBOXADDITIONS*
      sudo ./VBoxLinuxAdditions.run
      
  8. Reboot the VM
  9. Change the display resolution to something sane (System Settings -> Displays)
  10. If the Unity side taskbar annoys you too, make it auto-hide (System Settings -> Appearance -> Behavior; enable workspaces too) and install Cairo dock:
    sudo apt-get install cairo-dock
  11. Install Oracle the Java 8 JDK — not OpenJDK, because AndroidStudio doesn’t like that as much. You can do this easily thanks to the WebUpd8 team:
    sudo apt-add-repository ppa:webupd8team/java
    sudo apt-get update
    sudo apt-get install oracle-java8-installer
  12. If you’re running 64-bit Ubuntu, you’ll need to install a couple of 32-bit libraries the Android SDK needs to avoid the Unable to run mksdcard SDK tool” error:
    sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6
  13. Install Android Studio from Paolo Ratolo’s repo:
    sudo apt-add-repository ppa:paolorotolo/android-studio 
    sudo apt-get update 
    sudo apt-get install android-studio
  14. Run Android Studio and add it to the launcher:
    /opt/android-studio/bin/studio.sh
  15. If you need to, install git
    sudo apt-get install git