Skip to main content

The role of research -- Part I

My job is a research job, so a reasonable question is what is research in a corporate environment really for?. This is a two-part article; in this part I will concentrate on the problem. In the next part I will talk about the solution.

So, what is the problem of research?

If you ask an average person, even an average technical person, even someone who job is in research, what the role of research should be in an industrial context, my guess is that you will get an answer along the lines of

the job of research labs is to invent cool products for the business units to sell

Sometimes this is modified a bit:

the role of research labs is to support business units in whatever they want

I argue that this job description is both a poor mission for research labs and a poor use of the money spent.

First of all, consider an arbitrary business unit. Presumably, if they are doing their job correctly, they will have a constant eye out for products that they think they need - based on customer requests, gut feel, competitive landscape or all of the above.

I know that if I were a product manager in such a business unit, I would look first to my own people as a source of ideas and implementation for such missing products/functionality. I would only go somewhere else if I thought I couldn't do it internally. But, even if I did go outside, it would be my ideas that would drive the process. After all, I own the process; it's my responsiblity to ensure that my BU is a success.

I might go to a researcher if I had a particularly knotty problem; but most of the time the issues in developing products are not really all that hard. It is more a case of choosing the best available design compromise, tkaing into account developing technologies. The real issues in product design are to do with what customers you are targeting, what functionality you think that they will need and so on. Not rocket science generally.

So, what would make me go to the labs as a place to develop my ideas? As opposed, say, to going outside to another company or even to an ideas consultancy such as ideo? Certainly, if the labs had cool ideas I might consider them. But, as we shall see, the long term pressure on corporate research makes it an unlikely source for cool ideas.

One of the great problems with a product division using lab research directly is that their 'product' is hopeless from a productization perspective: I suspect that this is universally the case, but especially with software, a tool coming out of labs has to be rewritten from scratch to make it suitable. Its just too much trouble.

Also, typical researchers are very technology focused - they often 'get the bug' of a particular technology fix and spend their entire attention on promoting that particular hammer. For a hammer, everything is a nail. For a religious hammer, everything either is already a nail or should reconsider its position and become a nail forthwith. But that attitude is not very customer oriented; and it seriously limits the usefulness of the researcher to the BU.

Now consider from the lab's point of view. A typical researcher is a proud animal - he/she has worked hard to get a PhD. He is used to thinking deep thoughts. He doesn't want to have to spend time worrying about the user who spills coffee over his keyboard causing the database to suddenly start having a lot of wierd entries.

He typically doesn't even want to think about someone who cannot progam a VCR using his smart tool. But product managers have to think about who the customer really is, not what they would like the customer to be. If the customer cannot program a VCR, that just means that the VCR has to be smart enough to not need programming. (VCR manufactures take note!)

All that productization effort is not research - it does not forward the cause of knowledge - it does not help towards getting papers or patents. It is strictly a waste of the researcher's time and distracting from a careeer point of view.

This is not a recipe for success. In fact, I would argue that no research lab in modern times, especially a computer oriented lab, has been a successful partner for the main business when the lab's mission is expressed in terms of developing cool products.

I have witnessed many companies struggling with their labs. In fact, I have noticed a pattern:

When a company is flush with money, they decide to establish a research arm. If the choice is paying tax (or dividend) or starting a research arm, why not spend the money and get some use from it? There are many models for research arms; many ways of funding them etc. But almost all of them start by giving the researchers carte blanche - to think great thoughts that will ensure even greater success for the parent company.

Life is good as a researcher if you have carte blanche and a bunch of like minded fellows to share lunch with.

One thing to note at this point is that most people, researchers included, are much better at solving problems than at deciding which problem to solve. If you ask a researcher what the important problems are, your answer is not necessarily any better than asking any one else. Deciding what to do is much harder than just doing it.

Then, the money starts to slow down. At the same time, the expenses of research start to go up. Its natural - you have a good idea but it needs more investment to realize the potential.

The typical reaction is for some bean counter to declare that their research facility 'should be more accountable to the needs of the business'. Then what starts to happen is that the money that researchers get is no longer blue-sky but is more and more related to the goals of the business unit. Typically expressed in terms of products or technology that the BUs can market.

The managers of the BUs start getting pestered by researchers who need funding for their pet projects. The managers respond, typically, by treating the labs as outsourcing for their own product plans. The researchers end up becoming sub-contractors and being more and more product focused and less and less researchers.

Eventually, the BU managers will say that enough is enough - either you are part of my team (welcome aboard) or go away. The researchers might join the team (if they finally give up on their dream) or start polishing their resumes.

I personally have seen this progression in several Silicon Valley companies; as well as within my own. I have been pretty distressed by it all. Not because I think that it is wrong for research to support the goals of the company - far from it - but because I have seen that this model fails far more often than it succeeds. And because the net effect is mostly for the research labs to dissapear completely; having consumed vast resources along the way.

Something different is needed.

Something that respects the roles of the business unit, and of researchers. And that maximally leverages the natural instincts of the players.

I believe I have the answer for this; but you will have to wait for the answer :)

Popular posts from this blog

Comments Should be Meaningless

This is something of a counterintuitive idea: Comments should be meaningless What, I hear you ask, are you talking about? Comments should communicate to the reader! At least that is the received conventional wisdom handed does over the last few centuries (decades at least). Well, certainly, if you are programming in Assembler, or C, then yes, comments should convey meaning because the programming language cannot So, conversely, as a comment on the programming language itself, anytime the programmer feels the imperative to write a meaningful comment it is because the language is not able to convey the intent of the programmer. I have already noticed that I write far fewer comments in my Java programs than in my C programs.  That is because Java is able to capture more of my meaning and comments would be superfluous. So, if a language were able to capture all of my intentions, I would never need to write a comment. Hence the title of this blog.

Sub-turing complete programming languages

Here is an interesting intuition: the key to liberating software development is to use programming languages that are not, by themselves, turing-complete. That means no loops, no recursion 'in-language'. Why? Two reasons: any program that is subject to the halting problem is inherently unknowable: in general, the only way to know what a turing-complete program means is to run it. This puts very strong limitations on the combinatorics of turing-complete programs and also on the kinds of support tooling that can be provided: effectively, a debugger is about the best that you can do with any reasonable effort. On the other hand, a sub-turing language is also 'decidable'. That means it is possible to predict what it means; and paradoxically, a lot easier to provide a rich environment for it etc. etc. An interesting example of two languages on easier side of the turing fence are TeX and CSS. Both are designed for specifying the layout of text, TeX is turing complete and CSS

On programming languages and the Mac

Every so often I dig out my Xcode stuff and have a go at exploring developing an idea for Mac OS X. Everytime the same thing happens to me: Objective-C is such an offensive language to my sensibilities that I get diverted into doing something else. All the lessons that we have learned the hard way over the years -- the importance of strong static typing, the importance of tools for large scale programming -- seem to have fallen on deaf ears in the Objective-C community. How long did it take to get garbage collection into the language? I also feel that some features of Objective-C represent an inherent security risk (in particular categories) that would make me very nervous to develop a serious application in it. As it happens, I am currently developing a programming language for Complex Event Processing. Almost every choice that I am making in that language is the opposite to the choice made for Objective-C -- my language is strongly, statically typed; it is designed for parallel exe