One of the reasons I hear from scientists/engineers transitioning from wet work to “dry” work is that they would like to work in a field where, when things are fixed, they stay fixed. It’s not entirely clear to me that code, actual computer code that has to work in the field, completely fits that requirement and that folks should jump into computational work for that reason alone.
I have the opportunity, as part of the recruiting efforts of our company, to speak with folks at job fairs, conferences and during interviews. It takes me away from my desk, my plans and my deadlines, but it’s a nice change, and it’s one of the reasons I like working at the company (I get to wear different hats and I rarely have to wear one I don’t like, and I’ve rarely found a hat that I don’t at least want to try out – boy that hat analogy goes pretty far!)
I will often speak with folks who have an engineering or math background who are now doing wet work (experiments with biological systems) and want to transition out of wet work, and often out of academia, and into industry doing computational work. Been there, thought that, done that, very sympathetic to that.
Often the underlying reasons are complex (they were in my case) but sometimes they will manifest as a feeling of “I just fixed this rig YESTERDAY! Why doesn’t it record the data TODAY! Who $%#^ moved my ground wire?! Which #$%@^ idiot changed my filter settings?! I want to move to a field where, when I fix things THEY STAY FIXED!!”
Technically minded people who do both their own wet work and their computational/engineering (analysis, rig setup) work are falling prey to extinction. No, no this is not evolutionary extinction. This is “extinction” as a psychophysical phenomenon. This is when one stimulus is so salient (or you have a certain kind of brain damage) that it drowns out another stimulus, which exists, but is, for some reason, ignored by the brain, which focuses on the other stimulus.
In this case, technically minded people are magnifying the frustrations of working with biological and experimental systems over the frustrations of working on computer systems and this sometimes has to do with the nature of their setup.
In neurophysiology (and I imagine in most experimental work) we were pushing equipment to their limits. To the limits of signal transduction, to the limits of sampling, to the limits of their optics. As a result, they go beyond their engineering curve and become unreliable. The exact position of a ground wire starts to matter a lot more in your dreams (or nightmares) than seems reasonable. The fact that someone, two blocks away, is doing construction and is messing up recordings (by making a biological sample jiggle about 1 nano meter more than one would like) suddenly becomes a strain on a marriage. When you throw biological systems into the mix, well, that’s a whole another can of worms.
The computational aspect, on the other hand can be quite different. It’s challenging in the beginning, and in a good way. You need to write new code, debug it, (sometimes) test it with nasty looking data. You probably do it by yourself, so you know whats going on, you have it all in your head and it all flows out smoothly and creatively. It’s a lot of fun. And once you get it working, it’s a dream. It runs fine on your computer, and at most you may have to recompile it once to put it on your institution’s high performance computing cluster. It may be a few days work, because the software those jokers have on their machines is like a decade out of date, but in the end it is fixed, and it stays fixed.
In my opinion, this comparison is a little unfair to the wet work and it can be dangerous to draw too much by way of differences between wet work and dry work from this experience.
Computer programs that have been fixed stay fixed only when kept in stasis. You must never alter the hardware they are running on, you must never present them with new data and you must never let anyone use them. Real world computer programs don’t match this description.
One of the first things that will happen when you give this delicate, finely crafted, pure genius program that you have written to some one else, is that they will say “But it doesn’t work on WINDOWS! I spent all afternoon trying to download and compile HDF5, but it doesn’t WORK”.
In the off chance that it does work, you should never watch them trying to run the program. It’s a bit like being five and giving your favorite custom built LEGO model (of which you have no pictures and of course no plans) to your friend. The first thing they will do is drop it and it will smash into individual pieces all jumbled up on the floor. “So, I put in -1 where you asked for sequence length, and I got a core dump. Oh, it also erased <important data file>”
Then, this other person will come along, and want to work on your code. “It’s on <version control system>, right? I want to add <shiny feature>”. A few days later the program stops working. Or, even more likely, it works just fine, until this OTHER colleague comes along and says “My data look funny after they come out of your machine. Did you change anything?”
Writing test suites and documentation can be a chore. Why are you writing a test suite when you can be having fun throwing your machine at <big data>? Times-a-wastin. Why are you writing documentation and comments. If they are smart coders they’ll read the code and understand! It’s so BOOORING! I mean, it takes time away from other tasks.
Oh, and when everything is running and things are working fine, you’ll be asked to move your code to this big machine we have running in the cloud and everything will be going fine until you get this compilation error. And you have to spend the rest of the day looking high and low for which library needs to be updated to which version and which library has to be downgraded to which version before the the test suite passes. You have a test suite right? Because other wise a whole lot of OTHER people will be sending you email saying “My data looks funny after it comes out of your machine”.
The point is, none of these are unreasonable or improbable or malicious occurrences. They are a natural part of the process and if one does not like to deal with these aspects too, it will not be a joyful life.
Going back to the introduction and reasoning about career choices, I think, when figuring out what kind of work do, you should ask yourself, first, which matters more (and makes you happy): what you do or what you do it for? Is your job a means to an end, or an end in itself? Many people have judgements about this, all I can say is, everyone is different and you need to figure this out for yourself. Once you do this, the path becomes clear.
For me, working on computer code and algorithms has multiple pulls over lab work. The main ones are that I like learning new math and I LIKE wrestling with algorithms and bugs in my code. I like figuring out why my past me (the older the better) forgot a key aspect of how to tell the computer what to do, or forgot an interesting corner case that is now causing my program to fail in a baffling way. It makes for good war stories too. Compilation blockers can get annoying though …
Basically, I do it because I like the dirty parts as well as the shiny ones.