Another short story, where unit tests save my butt

I like to refactor. A lot. As I work on a problem I understand it better and I want to reflect this in the code. I was nearing the end of a pretty serious refactor. The tests had been failing for the last 60 commits or so. I wasn’t worried, I was expecting this. But now the time had come to resurrect the tests. I updated a bunch of the tests and was very gratified to see them pass with minor tweaks to match the new interface and architecture I had worked on. But then I got to a failure that was completely unexpected.

The code I was writing has meant to parse a file written in a particular language (CWL) and expose it’s logical structure. The failing test was a checking a completely different part of the code than what I was working on. It was reading in a test file written in CWL and claiming that all the connections were broken. The test claimed that the code should say the connections were there, and check for a particular connection.

It was like you were working on the spark timing of a car and when you were done the fuel pump stopped working.

I read over the CWL code and did a mental parse. It certainly LOOKED like the file was OK, and the test was correct, but I hadn’t altered this part at all. It was insulated from my refactor. So the test MUST be wrong. My code was working fine, I had tested it on dozens of other files.

Screen Shot 2019-03-24 at 5.14.04 PM.pngI opened up the test file in my refactored program. It indeed claimed that the connections between parts was broken. But I glanced at the raw CWL code on the right pane. Again my mental parse told me that the CWL program was fine. It was a simple program and hooked up the parts in a linear fashion. I should have lines going from input to node1 to node2 and finally to the output.

I was very eager to PASS myself. I was eager to move on. But the test was clocking me. I was getting angry at my test. Like a little child angry at it’s mother for saying no to the cookie.

I sighed and opened up the code that was responsible for this part of the parsing. The code that was not part of the refactor. The code my brain claimed I had not touched at all.

There, in the code responsible for this part of the processing was this line:

Screen Shot 2019-03-24 at 5.14.17 PM

I was suddenly rushed by a flood of recollection. It like those movies you know. The hero has amnesia, all sorts of things are going on, then he sees something, like a picture, or a person, and then you have these special effects that completely date the movie, and the hero remembers EVERYTHING.

In my new refactor – which I completely admit, was over-ambitious and thereby exposed couplings in the code I should have removed before attempting such an ambitious refactor – I had altered some code in unrelated classes in order to make them compatible with my new refactors. In this particular case I had made, effectively, a non-functional placeholder out of some key functionality, thinking “I’ll get back to this later.”

Screen Shot 2019-03-24 at 5.12.54 PM60 commits later, after all the other battles I had to fight during the refactor, I had apparently forgotten that I had buried a body here. Fortunately I had left a little trace in the code, in the form of that comment, which, fortunately, was enough to jog my memory. I changed the line to fit with the refactored code.

The test passed. And when I ran the test file through my whole program, I saw this diagram:

Which made eminent sense and was in line with what you would infer from a mental parse of the code on the right.

There were two things that sacred me however.

  1. Would I have figured out the problem as quickly as I had without my little note? It wasn’t even a big note – just the commented out code. That was bad. That was real bad. I should have written a note with a “todo” (the IDE I use, and several others, look for that string to help you find spots in your code left as patchwork) and an explanation of why I had done it. If I had someone else working on this code with me it might not have done.
  2. I was looking for a reason to not believe the unit test. “Oh, the program seems to work just fine!” I was trying to tell myself. “That document must really have a problem. My code is BETTER now, it’s probably finding an issue that I had missed before”. But I had worked hard on the unit tests. I had worked hard to make them cover the code AND to check the logic.

It was quite human of me to do both these things: I was eager to make the code better by refactoring it and I was eager to put everything together and move past the refactoring stage and start on the more interesting parts. It was scary never-the-less.

That is why we have processes and unit tests. To save us from ourselves.

So, I have two more rules now

  1. Leave extensive comments during patchwork. No one knows when some one is going to walk over that spot – mark it out clearly.
  2. Believe the unit test. Believe the unit test. Believe the unit test.

Okay, now that my knuckles have their color back, I see this pyramid has something called Integration tests and GUI tests. Some of my unit tests are integration tests – the space between those can be fluid when writing a standalone application – but GUI tests. Right now this is me sitting with my pre-written tutorial operating the final software. It’s too early to write GUI tests: the UI is still a little fluid. However, as the GUI settles down I should see about adding automated GUI tests. I understand QT makes it easy to do.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.