The Dark Side of Research
Photo by Nicolas Thomas on Unsplash
I wanted to talk about what a modern researcher does. What kind of researcher? There are academic researchers and informal researchers, but as a member of “infosec” will I mainly will be covering this from the perspective of a security researcher in information technology. Honestly there is a lot in common across different research disciplines, so this isn’t specifically aimed at infosec - that’s just my main perspective.
What I Mean Is…
I mentioned academic (systematic in gathering data via defined methods to prove or disprove a given hypothesis) and informal (less emphasis on rules and even a starting hypothesis) research, so maybe I should clarify that a bit first.
In many ways, the academic's payoff is the results while the informal's payoff is the journey. Let's look at a basic scenario - finding a needle in a haystack. The academic researcher will develop a scenario, like "what is the most efficient method of working one's way through a haystack to find the needle", they will take their haystack to a controlled space, possibly pull out each straw and noting its size and placement within the stack, document the most efficient way to decide how to work one's way from hay straws to the needle, and write up a detailed paper thoroughly covering the topic including the data gathering techniques. The informal researcher sets the haystack on fire, sifts the ashes, and pulls out the needle - and then writes up a paper on fire-starting techniques because that was the most interesting part for that researcher anyway.
But really, they are the same in many ways. If the researcher is happy and enjoys their work, that is really the main payoff. Truthfully I do not know anyone that is strictly academic or informal as every researcher I’ve ever met or know is somewhere in the middle of those two extreme examples.
That being said, there is a darker side to research, a side that those outside of the research world are largely unaware of. Most research is done using someone else's money, and sometimes the money controller has a specific opinion about what that research should cover, and sometimes they are paying for researcher and wanting a specific result. Academics face this when grant money is being used - if they don't produce something of interest to the funders, they might not get another round of grant money. Informals face this if their employer is expecting certain results, and those results are not met. The dreaded "return on investment" part of research is faced by all researchers everywhere.
Dark Scenarios
Here are a few examples of these dark sides that researchers face.
The first scenario is one I call "magic". This is where on a scale of 1 to 10 with 10 being the most complex thing ever, the funder (the person paying the bills) understands 1 through 3 okay, but 4 and above are simply "magic" as far as they are concerned. This creates a bit of a problem when you can deliver a 4 or 5 in a couple of months, but delivering a 9 might easily be a year long effort. Let’s say the funder asks for some specific research and expects completion in a couple of months, like the last thing you worked on. Trying to explain that this new thing is a 9 and the old thing was a 4 is going to be kind of lost on the funder, but when you're in magic territory this can create massive problems - especially if the funder sees no difference between a 3 and a 10. If this is your job, you can simply pump out 3's and make your boss happy and fight for the occasional and rare magic research project, or you can seek employment or grant money elsewhere.
The second scenario is the unwanted output scenario. You are given a task, and you reach a conclusion that is by all measurements complete. However the funder doesn't want or even agree with the conclusion, and therefore dismisses the entire scenario as a waste of time and money. I once looked at a piece of technology, and while I found the core of the product secure and the story of how secure it was to be interesting as did my boss, my boss' boss did not agree. Not finding a security bug was apparently boring and a waste of time. Forget the fact that the methodology I came up with was interesting, and that I ended up applying the exact same methodology on future projects which yielded "positive" results, the secure thing was boring and not published. I purposely did not list what product this was, because this happened to me at more than one employer with more than one immediate boss delivering the bad news from up the management chain that no, boring, do something else, you wasted time and money.
The third scenario is the dreaded "success by others' failures, with repercussions" scenario. This is where you are either building upon previous work, or you find something wonderfully astounding and fruitful but someone else is going to look bad. I once found some bugs in a software component that my employer used as a part of our own product offering. This component was not written by my employer, but was licensed from this other software company. The bugs were deliciously hackerish, we could remotely take over systems running our flagship product. One of us found it, and several of us helped out and built upon it to make it ever so evil. Our bosses loved it - we found something exciting, and we were improving our product at the same time. When we contacted the company we licensed this component from, they went ballistic. Why would you do this? Why would someone want to take over systems? Isn't this illegal? We explained that we were only going public with it after it was patched ("you're going PUBLIC?!?!"), tried to explain full disclosure and so on, but it required massive interference from upper management talking to this other company to calm things down and prevent something ugly involving lawyers. This type of research was never done again at that employer.
The fourth scenario is the "magic means nothing" scenario. Similar to the magic scenario, you've found a 9 and in spite of the fact that your bosses have no idea how you reached this, they at least understand the result enough to say "awesome, this is great" and you release your research. Perhaps 10 people on the planet actually give a shit, and apparently only 5 even acknowledged the work. The lackluster response sours your bosses into pursuing similar work. This happened to me several jobs ago, where despite everyone at work being excited about it, it fell flat. Even worse, the bug I found had such interesting implications from a security perspective that the vendor could not even patch it - they simply recommended the affected buyer of the vendor's buggy software purchase an expensive add-on module. Home run, out of the park, but the park was empty. No one cared. Yes, a few people came forth and congratulated me, and I was even courted for a position at a U.S. government spy agency to do even more work in that area (for like half the pay, public sector back then paid even worse than they do now), but it was by all measurements of my employer unsuccessful. If the grant provider or employer expects the world to go crazy and nothing seems to happen, yeah, they are probably not thrilled if you head down that path again.
The fifth scenario is when an organization simply pays money to come up with "research" to support a specific outcome. This is the worse thing of all. My fellow researchers in infosec will say, oh that only happens in academic circles and via the academic methods, not here in infosec hacker land. While you can easily find such academic examples when the desired outcome involves smoking dangers, asbestos in baby powder, or poisoned water supplies, know this - they do exist in informal research as well. I have personally been approached and offered what was at the time a year's worth of salary to find a flaw in the software of a competitor by an extremely large organization. I have seen a large software organizations threaten to pull co-op marketing dollars from a campaign, and then not-so-casually mention they would not pull the marketing dollars if "that research department of yours would stop making us look bad".
Summary
This sounds horrible, right? Actually it is not that bad. This is something that all researchers deal with to a certain degree, and of course the best way to deal with it is to celebrate those victories when you get them, and try not to take it personal when things don’t go your way.
As a speaker at security events, I have had the good fortune to have conversations with academic researchers, informal researchers, and those that live with feet in both worlds. Often these conversations take place at said security events, but also at dinner or late at night in a dimly lit bar. All researchers have confirmed these scenarios, and often it is one of those major contributing factors to "burnout" that many people in infosec bitch about via social media. It can wear you down if you let it.
While I am technically a researcher at my current employer (GitLab), my scope is my employer's domain - the product, the platform it exists on, and any and all related technology in use. I am also fortunate that the company is filled with tech nerds all the way up the management chain that know the difference between a 3 and a 9. With my employer being extremely open about everything, the problems of the above scenarios have no way to even surface, let alone exist. Of course I currently don’t do that form of stunt hacking where you find flaws in an IoT power drill (yes I did that), but I can flex my research muscles quite enthusiastically and effectively where I am now, thank you very much. I even have the freedom to pursue those smart power drills or toasters or medical devices or whatever if I wish to do so on my own time (I may still), with my employer's blessing.
I do feel lucky, like I made it out the other side or something, and am relatively unscathed. But I always keep in mind the dark side of research, if for no other reason than to support those that fight the good fight.
