Why deep learning will widely transform the programming.

Dedication. To my father, Vladimir Mitin, for his interest in technology and science that formed my way of thinking.

Considering that Deep Learning is a universal function approximator, it has a good chance to become a new universal approach for development.

Let’s highlight techniques helping to scale traditional development processes.

First group of such techniques are architectural, technical and organizational methodics aimed to break tasks down, arrange separate execution of sub-tasks and combine the results in a workable solution.

Second group is everything related to knowledge accumulation, transfer and reuse.

Third group is all sorts of tools amplifying and boosting the programming process itself.

The boundaries of second and third group tools are permanently expanding. As an illustration, just compare frameworks and tools nowadays and ten, twenty, thirty years ago. The progress of the first group, that is the main way of arranging development scalability, is noticeably slower.

Weak scalability is the main limitation of the traditional software development process. The maximum efficiency is achieved for scope that could be done by one developer. Once you are in need to have more than one developer, you start losing efficiency dramatically.

What are the roots of this weakness?

1. Speed and quality of data exchange between developers.

2. Combining developed parts together.

Deep Learning revolutionizes programming by replacing “strong” human developer with “weak” computational “developer”. This weak developer programmes an elementary algorithm capable of combining multiple inputs linearly and applying some non-linearity to the result. In particular, in order to achieve the goal, it programmes the weights of these transformations. In other words, the weights are the code generated by this programmer. What is more important, DL combines many algorithms, each solving it’s own atomic task, into a tremendous ensemble capable of solving complex tasks. That eliminates traditional development paradigm bottlenecks by simple and beautiful mechanics of forward propagation of results and backward propagation of errors.

As a result, instead of slowly and weakly communicating strongly intelligent human developers, creating human readable code, we’ve got fastly and strongly communicating weak algorithmic developers, creating code as a set of weights.

Instead of tens, hundreds strong developers in a team, we’ve got the possibility to form fastly and widely linked “team” of hundreds, thousands, tens thousands, hundreds thousands, millions, tens millions and so on weak developers.

That constitutes a completely different level of flexibility in development resource allocation and in the orders of magnitude higher and continually increasing level of development scalability.

No surprise that Deep Learning massive breakthroughs happened in recognition, the tasks that are extremely complex and require big amounts of development resources. It will not be a surprise that DL will spread to other programming areas. Let’s remember it’s universality as a function approximator. Let’s remember that DL fastly accumulates, transfers and reuses knowledge. Let’s remember that DL evolves and improves it’s machine learning process.

Finally, let’s have a look how human developers roles will be transformed by this technological shift. Traditional programming is based on implementing in code the rules that are learned by the human mind from common sense, domain specific knowledge and task specification. DL programming is based on implementing in model weights rules that are learned by model out from data provided. It means that there should be an information engineering role ensuring that the model will receive data containing knowledge to be extracted. It could be preprocessed existing data or it could be specially generated data. The next role will be a deep learning engineering role, finding appropriate model architecture and/or pretrained models and defining appropriate training approaches to train the model extracting the knowledge. The final role will be inference engineering role, creating a model for inference based on a model trained by deep learning engineering role. For some tasks it will require simplifying the model, for some tasks it will require extracting the knowledge learned by model in explicit form and so on.