Skip to main content

The Role of Research Part II (Innovate or die)

When I was growing up, the overwhelming reaction I got from people from all walks of life was that change was something to be avoided. I found it frustrating because, to me, it seemed that there was so many exciting things that could be happening but noone was interested.

That has changed today. Nowadays, it is accepted wisdom that you must innovate - or suffer the consequences.

Providing services, providing products

We can classify, roughly, a business into two primary styles - a service provider or a product provider. Be aware that the line between these can get pretty fuzzy. However, a key difference is in the nature of the relationship with the customer: a service provider 'takes control' of the relationship by offering to directly meet some requirements. A product provider is also focused on meeting requirements; but is more indirect: the product provider sells a tool to the customer that enables him or her to meet his or her requirements.

There are other differences between the two, but our focus here is on the styles and modes of innovation applicable to each. And the resulting implications for research efforts.

The primary locus of innovation for a service provider is in production - in the processes involved in delivering the service. For example, in a Systems Integration business, the primary point where innovation must happen is in the process of capturing requirements, constructing specifications, implementing systems, testing, deployment and customer management. Of these, of course, the hardest to get right, and the most critical, is the first - capturing requirements. Get that wrong and the whole exercise becomes expensive very rapidly.

For a product provider, the story is more complex. Because a product provider delivers a tool to the customer, whether its a mobile phone gadget or an Enterprise suite, there are two primary loci of innovation: in the product itself and in the process for manufacturing and marketing that product.

Product providers are often organized around the products that they develop and sell; with separate units focused on separate products. A product manager is tasked with ensuring that the products meet customers' requirements and can be manufactured at reasonable cost. For software products, the manufacturing process itself is trivial, the main costs are in development (i.e., research), in marketing the product and in supporting customers' use of the product. Since most of the upfront investment in a software product is in development, that is a critical point for a software product manager to focus on.

In an environment of change, especially one where change is actually accelerating, it is important that the management of a product provider is aggressively evolving the product offering as well. This means that a well managed product provider has a stream of new products coming out; each hopefully better than the previous and each better able to solve customers' needs.

That, in turn, seems to suggest that the real character of a product provider is one of a process of developing new products! Except that the customer for this process is the company itself. The management of the product provider must be geared to the process of constructing new products. If the management is effective, then they are the best placed team to solve the problem of developing new products.

Product research is best not done in centralized labs

This one of the fundamental reasons why research labs are not good places to develop new products: that job is primary responsibility of the managers of the product and service providers! And trying to do product development in research labs puts them in conflict with the product providers.

On the other hand, anything that is not directly connected with the next product is fundamentally a distraction for the product manager. He or she is focused on putting together the pieces necessary for the next project. He will take in any number of inputs and requirements, so long as they relate directly to the product at hand.

This is one of the reasons why the product development environment is not a good locus for tool development. It is a well known fact of management that developing infrastructure tools generally slows down the development of the project the tools will be used for.

For example, consider the problem of developing a word processor. Now, imagine a time long ago when there were no object oriented programming languages. The manager of the word processor product might say: “I wish that we had an OO language, then it would be much easier to develop our word processor”. He might be right; but if he then goes on to develop resources into building that OO system, he will inevitably slow down the development of the word processor itself!

In general, the manager of the word processor team must try to minimize any development that is not directly focused on the word processor itself. Even to the point of saying that “even if an OO language would make my WP much more reliable, faster, etc., I can't spend any resources developing the OO language because that is not my task”.

Note that this includes the deployment of tools coming outside the team: the OO language might exist but if noone in the WP group knows how to use it then that can easily slow the development cycle down.

However, and this is the kicker for the WP team manager: he has to be willing to change his product development process if he is going to remain competitive with other WP manufacturers. It is just that the natural energy for developing products is contrary to the adoption of and especially the development of general tools.

So, the take home message is that the proper place for developing new products is inside product units, and that centralized research labs are inherently poor places to develop products. Conversely, product units are not very well placed to develop infrastructure. There is a glimmer of a rational structure here.

In the next (and final) installment, I will try to outline what I think is a rational basis for organizing technical research in a commercial venture.

Popular posts from this blog

Comments Should be Meaningless

This is something of a counterintuitive idea: Comments should be meaningless What, I hear you ask, are you talking about? Comments should communicate to the reader! At least that is the received conventional wisdom handed does over the last few centuries (decades at least). Well, certainly, if you are programming in Assembler, or C, then yes, comments should convey meaning because the programming language cannot So, conversely, as a comment on the programming language itself, anytime the programmer feels the imperative to write a meaningful comment it is because the language is not able to convey the intent of the programmer. I have already noticed that I write far fewer comments in my Java programs than in my C programs.  That is because Java is able to capture more of my meaning and comments would be superfluous. So, if a language were able to capture all of my intentions, I would never need to write a comment. Hence the title of this blog.

Sub-turing complete programming languages

Here is an interesting intuition: the key to liberating software development is to use programming languages that are not, by themselves, turing-complete. That means no loops, no recursion 'in-language'. Why? Two reasons: any program that is subject to the halting problem is inherently unknowable: in general, the only way to know what a turing-complete program means is to run it. This puts very strong limitations on the combinatorics of turing-complete programs and also on the kinds of support tooling that can be provided: effectively, a debugger is about the best that you can do with any reasonable effort. On the other hand, a sub-turing language is also 'decidable'. That means it is possible to predict what it means; and paradoxically, a lot easier to provide a rich environment for it etc. etc. An interesting example of two languages on easier side of the turing fence are TeX and CSS. Both are designed for specifying the layout of text, TeX is turing complete and CSS

On programming languages and the Mac

Every so often I dig out my Xcode stuff and have a go at exploring developing an idea for Mac OS X. Everytime the same thing happens to me: Objective-C is such an offensive language to my sensibilities that I get diverted into doing something else. All the lessons that we have learned the hard way over the years -- the importance of strong static typing, the importance of tools for large scale programming -- seem to have fallen on deaf ears in the Objective-C community. How long did it take to get garbage collection into the language? I also feel that some features of Objective-C represent an inherent security risk (in particular categories) that would make me very nervous to develop a serious application in it. As it happens, I am currently developing a programming language for Complex Event Processing. Almost every choice that I am making in that language is the opposite to the choice made for Objective-C -- my language is strongly, statically typed; it is designed for parallel exe