Skip to main content

The role of research -- Part I

My job is a research job, so a reasonable question is what is research in a corporate environment really for?. This is a two-part article; in this part I will concentrate on the problem. In the next part I will talk about the solution.

So, what is the problem of research?

If you ask an average person, even an average technical person, even someone who job is in research, what the role of research should be in an industrial context, my guess is that you will get an answer along the lines of


the job of research labs is to invent cool products for the business units to sell


Sometimes this is modified a bit:

the role of research labs is to support business units in whatever they want


I argue that this job description is both a poor mission for research labs and a poor use of the money spent.

First of all, consider an arbitrary business unit. Presumably, if they are doing their job correctly, they will have a constant eye out for products that they think they need - based on customer requests, gut feel, competitive landscape or all of the above.

I know that if I were a product manager in such a business unit, I would look first to my own people as a source of ideas and implementation for such missing products/functionality. I would only go somewhere else if I thought I couldn't do it internally. But, even if I did go outside, it would be my ideas that would drive the process. After all, I own the process; it's my responsiblity to ensure that my BU is a success.

I might go to a researcher if I had a particularly knotty problem; but most of the time the issues in developing products are not really all that hard. It is more a case of choosing the best available design compromise, tkaing into account developing technologies. The real issues in product design are to do with what customers you are targeting, what functionality you think that they will need and so on. Not rocket science generally.

So, what would make me go to the labs as a place to develop my ideas? As opposed, say, to going outside to another company or even to an ideas consultancy such as ideo? Certainly, if the labs had cool ideas I might consider them. But, as we shall see, the long term pressure on corporate research makes it an unlikely source for cool ideas.

One of the great problems with a product division using lab research directly is that their 'product' is hopeless from a productization perspective: I suspect that this is universally the case, but especially with software, a tool coming out of labs has to be rewritten from scratch to make it suitable. Its just too much trouble.

Also, typical researchers are very technology focused - they often 'get the bug' of a particular technology fix and spend their entire attention on promoting that particular hammer. For a hammer, everything is a nail. For a religious hammer, everything either is already a nail or should reconsider its position and become a nail forthwith. But that attitude is not very customer oriented; and it seriously limits the usefulness of the researcher to the BU.

Now consider from the lab's point of view. A typical researcher is a proud animal - he/she has worked hard to get a PhD. He is used to thinking deep thoughts. He doesn't want to have to spend time worrying about the user who spills coffee over his keyboard causing the database to suddenly start having a lot of wierd entries.

He typically doesn't even want to think about someone who cannot progam a VCR using his smart tool. But product managers have to think about who the customer really is, not what they would like the customer to be. If the customer cannot program a VCR, that just means that the VCR has to be smart enough to not need programming. (VCR manufactures take note!)

All that productization effort is not research - it does not forward the cause of knowledge - it does not help towards getting papers or patents. It is strictly a waste of the researcher's time and distracting from a careeer point of view.

This is not a recipe for success. In fact, I would argue that no research lab in modern times, especially a computer oriented lab, has been a successful partner for the main business when the lab's mission is expressed in terms of developing cool products.

I have witnessed many companies struggling with their labs. In fact, I have noticed a pattern:

When a company is flush with money, they decide to establish a research arm. If the choice is paying tax (or dividend) or starting a research arm, why not spend the money and get some use from it? There are many models for research arms; many ways of funding them etc. But almost all of them start by giving the researchers carte blanche - to think great thoughts that will ensure even greater success for the parent company.

Life is good as a researcher if you have carte blanche and a bunch of like minded fellows to share lunch with.

One thing to note at this point is that most people, researchers included, are much better at solving problems than at deciding which problem to solve. If you ask a researcher what the important problems are, your answer is not necessarily any better than asking any one else. Deciding what to do is much harder than just doing it.

Then, the money starts to slow down. At the same time, the expenses of research start to go up. Its natural - you have a good idea but it needs more investment to realize the potential.

The typical reaction is for some bean counter to declare that their research facility 'should be more accountable to the needs of the business'. Then what starts to happen is that the money that researchers get is no longer blue-sky but is more and more related to the goals of the business unit. Typically expressed in terms of products or technology that the BUs can market.

The managers of the BUs start getting pestered by researchers who need funding for their pet projects. The managers respond, typically, by treating the labs as outsourcing for their own product plans. The researchers end up becoming sub-contractors and being more and more product focused and less and less researchers.

Eventually, the BU managers will say that enough is enough - either you are part of my team (welcome aboard) or go away. The researchers might join the team (if they finally give up on their dream) or start polishing their resumes.

I personally have seen this progression in several Silicon Valley companies; as well as within my own. I have been pretty distressed by it all. Not because I think that it is wrong for research to support the goals of the company - far from it - but because I have seen that this model fails far more often than it succeeds. And because the net effect is mostly for the research labs to dissapear completely; having consumed vast resources along the way.

Something different is needed.

Something that respects the roles of the business unit, and of researchers. And that maximally leverages the natural instincts of the players.

I believe I have the answer for this; but you will have to wait for the answer :)

Popular posts from this blog

Concept Oriented Markup

I have long been frustrated with all the different text mark up languages and word processors that I have used. There are many reasons for this; but the biggest issue is that markups (including very powerful ones like TeX) are not targeted at the kind of stuff I write.

Nowadays, it seems archaic to still be thinking in terms of sections and chapters. The world is linked and that applies to the kind of technical writing that I do.

I believe that the issue is fundamental. A concept like "section" is inherently about the structure of a document. But, what I want to focus on are concepts like "example", "definition", and "function type".

A second problem is that, in a complex environment, the range of documentation that is available to an individual reader is actually composed of multiple sources. Javadoc exemplifies this: an individual library may be documented using Javadoc into a single HTML tree. However, most programmers require access to multiple…

Existential Types are the flip side of generics

Generic types, as can now be seen in all the major programming languages have a flip side that has yet to be widely appreciated: existential types.

Variables whose types are generic may not be modified within a generic function (or class): they can be kept in variables, they can be passed to other functions (provided they too have been supplied to the generic function), but other than that they are opaque. Again, when a generic function (or class) is used, then the actual type binding for the generic must be provided – although that type may also be generic, in which case the enclosing entity must also be generic.

Existential types are often motivated by modules. A module can be seen to be equivalent to a record with its included functions: except that modules also typically encapsulate types too. Abstract data types are a closely related topic that also naturally connect to existential types (there is an old but still very relevant and readable article on the topic Abstract types have …

Robotic Wisdom

It seems to me that one of the basic questions that haunt AI researchers is 'what have we missed?' Assuming that the goal of AI is to create intelligence with similar performance to natural intelligence; what are the key ingredients to such a capability?

There is an old saw
It takes 10,000 hours to master a skill
There is a lot of truth to that; it effectively amounts to 10 years of more-or-less full-time focus. This has been demonstrated for many fields of activity from learning an instrument, learning a language or learning to program.

But it does not take 10,000 hours to figure out if it is raining outside, and to decide to carry an umbrella. What is the difference?

One informal way of distinguishing the two forms of learning is to categorize one as `muscle memory' and the other as 'declarative memory'. Typically, skills take a lot of practice to acquire, whereas declarative learning is instant. Skills are more permanent too: you tend not to forget a skill; but it is…