Formerly known as Wikibon
Close this search box.

Augmented Programming Brings Machine Learning Into The Development Toolchain


Augmented programming drives code generation through high-level development abstractions. One of the more promising recent innovations in augmented programming is the use of machine learning (ML) algorithms to greatly reduce the need for hand coding. Today’s ML-driven augmented programming solutions are the forerunners of future tools that will automate the application development lifecycle to an unprecedented degree.

Expert human judgment remains the core of high-quality software development. Nevertheless, programmers will continue to adopt tools to accelerate, automate, or eliminate all or most repetitive tasks at every stage in the development lifecycle.

Simplify, Simplify, Simplify

For many years, software developers have used code generation tools to lighten the load. Some refer to these as “automatic programming” solutions, though you’d be hard-pressed to name any developer who’s ever automated him or herself out of a job. Another popular phrase for these tools is “low code” or even “no code.” However, those terms obscure the fact that they all generate lots of source code and usually give programmers tools for maintaining and revising it all.

Augmented programming refers both to established and emerging approaches for boosting developer productivity. One of the hottest new approaches under this umbrella is robotic process automation (RPA). The common thread in all augmented programming tools is use of an abstraction layer that allows developers to write declarative business logic that is then translated by tools into procedural programming code.

Increasingly, RPA and other augmented programming tools use ML as an abstraction layer for automatically inferring program code from screenshots and other existing application elements. By reverse-engineering program code from existing applications, ML can boost developer productivity to an extent that traditional code-generation tools–with their reliance on manual techniques for declarative specification—have been unable to deliver.

Figure 1 shows the three core technologies that drive augmented programming.

Figure 1: The Core Technologies of Augmented Programming

When considering how best to augment their programming practices, developers should heed the following Wikibon guidance:

  • Determine whether to augment programming across all development projects or simply in particular application domains. Developers should identify the degree to which your augmented-programming initiative should address a narrow or broad range of development requirements. Some tools are limited to such development approaches as e-form and database applications, while others address a wider range of Web, workflow, mobile, analytics, and case management requirements.
  • Adopt robotic process automation to augment programming of office applications by non-traditional developers. Explore the use of RPA software “robots” to infer an application’s underlying logic from its presentation layer, user-interface controls, interaction and messaging flow, and application programming interfaces. In this regard, the robots rely on ML, deep learning (DL), natural language processing (NLP), and computer vision to infer source code from externally accessible program elements
  • Explore emerging tools that rely exclusively on machine learning for automated program-code generation. Though immature as a development tool, ML-augmented coding is likely to gain adoption as a rapid application development tool that supplements and extends enterprise investments in low-and RPA tools. Rather than “re-invent the wheel” with handcrafted code or repurposed code modules, future developers may simply check off program requirements in a high-level GUI, and then, with a single click, auto-generate the predictively best-fit code-build into the target runtime environment.

Determine Whether To Augment Programming Productivity Across All Development Projects Or Simply In Particular Application Domains

Augmented programming solutions enable developers to visually design, build, customize, and deploy application software with little or no coding.

These tools provide abstractions that enable developers to craft declarative specifications that will drive automated generation of application source code. Table 1 presents the most common abstractions in augmented programming.

Domain-specific languages These provide a syntax for programmers to specify the application logic associated with a particular domain, such as advanced analytics, content markup, graphics, and mathematics.
Scripting languages These support specification of application-level tasks that can be automated, individually or as complex programs, as an alternative to having them executed one-by-one by human users, operators, or administrators.
Database models These enable programmers to define a database’s logical structure, including the key relationships among separate tables, and the operations that may be performed on data elements.
Metadata models These enable definition of the concepts, frames, rules, constraints, and other metadata associated with the modeling of specific classes of programming challenges, such as neural networks, process engineering, and data manipulation.
Flowchart models These support diagrammatic specification of workflows and other application level constructs in a directed graph of steps with dependencies, rules, and constraints.
Graph models These enable description of business logic as links among processes, tasks, devices, people, and other entities.
Tree models These support specification of abstract data types and structures in a hierarchical structure, with a root value, a parent node, and subtrees of references to linked child nodes.
Templates These allow programmers to write declarative business logic and automatically instantiate that specification at compile time into functions, data structures, and constants.

Table 1: Declarative Abstraction Layers in Augmented Programming

Where Machine Intelligence Fits

Increasingly, the abstraction layer in many tools also incorporates ML and other artificial intelligence (AI) approaches to drive algorithmic source-code generation. Essentially, ML/AI enables specification of which variables in application data map statistically to programmatic behaviors, such as making predictions and identifying environmental anomalies.

Regardless of the underlying programming abstraction, all augmented-programming tools produce well-formed textual statements in various underlying procedural and markup languages, such as Java, C#, C++, VB.NET, Python, SQL, JSON, JavaScript, XML, HTML, and so on. Figure 2 provides a graphical overview of how augmented programming tools work.


Figure 2: How Augmented Programming Works

Many vendors provide augmented programming tools, often under the heading of “low code” solutions (though “no code” and “rapid application development” are often used interchangeably describe this segment). Table 2 lists key solution providers in the low-code segment and highlights the chief application focus areas of their respective solutions.

Adobe Mobile, web
AgilePoint E-forms, Web, SaaS, Mobile, BPM
Alpha Software Mobile, web
Alphinat Web, cloud, mobile Mobile, desktop Mobile
Appian BPM
Automation Anywhere BPM
BettyBlocks Mobile, portal, web, enterprise
Bizagi BPM
BlackLine BPM
Blue Prism BPM
BP Logix BPM
Caspio Database, e-forms, web
ClaySys Technologies BPM, e-forms, web
CommonTime Mobile, BPM, web
Dell Boomi BPM
EASA Software Web
Elevate Web
Formotus Mobile, e-forms
Google Enterprise
IBM Mobile
K2 BPM, e-forms
King of App Mobile
Kintone Mobile, web, e-forms, enterprise
Kofax BPM
Kony Mobile
LANSA Web, mobile, e-forms, desktop
Linx Web, mobile, e-forms, desktop, enterprise
MatsSoft Web, mobile, e-forms, desktop, enterprise
Mavo Web
Mendix Web, mobile, e-forms, desktop, enterprise
MicroPact BPM, case management, mobile
Microsoft Web, mobile, e-forms, desktop, enterprise
MobileSmith Mobile
Nintex BPM, e-forms
Omnis Studio Desktop, web, mobile
Oracle Web, mobile
OutSystems Mobile
Pegasystems BPM
PNMsoft Mobile, BPM
Progress Web, mobile, e-forms, desktop, enterprise
QuickBase Web, mobile, e-forms, desktop, enterprise
Rapidsoft Web, mobile
Salesforce Web, mobile, e-forms, enterprise
ServiceNow BPM, case management, e-forms, Web, mobile, enterprise
Simplicité Software Web, mobile
Software AG BPM, e-forms, Web, mobile, enterprise
Spring Web
Torus BPM, e-forms, Web, mobile, enterprise
TrackVia BPM, e-forms, mobile, Web
UiPath BPM
Ultimus BPM, e-forms, mobile, Web
Vantiq BPM, case management, e-forms, Web, mobile, enterprise
ViziApps Mobile
WaveMaker Web, mobile
WhoGloo Web, mobile
Xojo Desktop, mobile, Web
Zoho BPM, e-forms, Web, mobile, enterprise
Zudy Web, mobile, enterprise

Table 2: Principal Low-Code Development Tool Vendors

Here’s Forrester Research’s John Rymer discussing low-code programming on the Cube in 2016.

Evaluation Criteria for Augmented Programming

When evaluating augmented programming solutions in the low-code segment, developers should consider the criteria in Table 3.

Application and runtime domains Many low-code tools are general-purpose application builders, while others are specialized to Web, mobile, business process management, and other application domains. The tools enable single-click deployment of the auto-generated procedural code, as well as associated metadata, for execution in various target environments, including on-premises, cloud, and other mobile platforms. The auto-generated procedural code—typically in C#, C++, Java, JavaScript, PHP, or another general-purpose language–may serve as basis for upfront customization and downstream maintenance. Many tools also support various domain-specific languages for specialized line-of-business applications.
Skill levels In addition to supporting the production requirements of professional software developers, many such tools allow less skilled line-of-business professionals to prototype their own apps without having to write procedural code. In this latter regard, these tools are an enterprise-grade alternative to traditional do-it-yourself development techniques such as Visual Basic programming in Microsoft Excel. Some tools provide guidance to help developers accelerate achievement of their programming tasks, while others provide far less handholding and assume that the user is a skilled developer.
Programmer experience To support self-service programming, low-code tool suites provide development experiences that are visual, declarative, and model-driven. Just as with the fourth-generation programming language and rapid application development tools from which they evolved, they use drag-and-drop WYSIWYG interfaces, form builders, on-screen objects, and visual workflows to drive source-code auto-generation. Developers use these features to specify what they want to happen rather than prescribing the steps the computer should take.
Development libraries To support rapid development, low-code tools typically include reusable code libraries, scripts, models, connectors, orchestrations, templates, forms, object mappings, and other development artifacts associated with various application domains. They give developers tools for visually defining apps’ data models, business logic, workflow processes, and UIs. Likewise, they integrate with databases, web services, and APIs associated with those applications domains.
Ecosystem integration Through their own and partner tooling, many low-code suites also support developers’ ongoing efforts to extend, customize, refactor, containerize, orchestrate, migrate, and optimize legacy code and markup for cloud-native application environments. Likewise, most incorporate features and/or interfaces to partner DevOps tools for continuous integration, issue tracking, schema migration, monitoring, security, and scaling associated with auto-generated application code.
Cost of ownership Pricing for low-code tools varies widely by vendor. However, per-user monthly pricing is most common, with the price often depending on concurrent users, number of apps, license types, and components.

Table 3: Low-Cost Augmented-Programming Solution Considerations

Adopt Robotic Process Automation To Augment Programming Of Office Applications By Non-Traditional Developers

Robotic process automation (RPA) auto-generates source code and other program elements from externally observable application artifacts and behaviors. RPA tools converge augmented programming with ML, business process orchestration and Web content management.

Common applications of RPA “software robots” include automation of the following:

  • Authentication: logging into applications
  • Data management: reading and writing to databases, extracting structured and unstructured data from the web and documents, merging data from multiple sources, copying and pasting data
  • Forms: filling in and submitting e-forms
  • Messaging: opening emails and attachments
  • File management: moving files and folders
  • Programmatic operations: connecting to system APIs, following deterministic rules, making calculations

Chief RPA solution providers include Automation Anywhere, BlackLine, Blue Prism, Kofax, Pegasystems, and UiPath. Check out this recent discussion by UIPath’s Bobby Patrick on the Cube, where he spells out RPA’s value proposition.

Currently, RPA solutions are not a full replacement for general-purpose, low-code tools. That’s because today’s RPA are geared primarily to development of Web and mobile application code through algorithmic inferences from screenshots, schemas, workflow definitions, and other existing assets. They are not full low-code development suits with features suited to continuous integration, issue tracking, migration, monitoring, security, and scaling of code-builds for a wide range of application domains.

Evaluation Criteria for RPA

When evaluating augmented programming solutions in the RPA segment, developers should consider the criteria in Table 4.

Application complexity Essentially, RPA software “robots” infer an application’s underlying logic from its presentation layer, user-interface controls, interaction and messaging flow, and application programming interfaces. In this regard, the robots rely on ML, DL, NLP, and computer vision to infer source code from externally accessible program elements. Key RPA capabilities include screenscraping of UI presentation elements, optical character recognition of on-screen text, auto-sensing of browser-level control and domain object models, recording of human-user keystrokes and clicks, and user-specified flowcharting of UI flows.
Reverse-engineering or greenfield development These capabilities enable RPA tools to either to automatically build a high-fidelity replica of an existing application, or generate new apps from prototypes that users may craft in the form of screens, wireframes, database schemas, workflows, and other program elements. As with screenscraping tools, RPA robots interpret the UI of applications and can infer how to execute application steps identically to those that a human user might take.
Target execution environment These software robots may be installed on desktops, servers, and other application nodes. As their footprint expands throughout a distributed application environment, RPA robots may be configured to observe, capture, profile, and reproduce any flow of content, context, and control, whether it be on the front-end, back-end, or distributed architectures. They can also be setup to non-disruptively and automatically detect changes to APIs, document object models, and other observable application architectures, then triggering changes to auto-generated application code that interface to these features. The robots themselves can usually be set up to execute code in entirely unassisted automations or with varying degree of human supervision and assistance.

Table 4: RPA Augmented-Programming Solution Considerations

Figure 3 presents a graphical overview of how RPA solutions work.

Figure 3: Robotic Process Automation in Action (Source: Deloitte: “Automate this: The business leader’s guide to robotic and intelligent automation”)

RPA might unnerve people who’ve made their careers writing the application code and orchestration logic that underpins many enterprise applications. In fact, many RPA implementations come into organizations throughout business operations who are frustrated waiting for IT shops to deliver on their complex application requirements and who want to take the matter in their own hands. Indeed, RPA robots may be able to produce a first cut on these auto-built orchestrations better and faster than an expert human process designer. This is, in fact, how many organizations are using RPA, providing flowcharting tools that both traditional developers and business users can use to graphically monitor and tweak the process definitions that distributed RPA infrastructures automatically generate.

One might regard RPA as the Trojan Horse of the “citizen programmer.” One of the technology’s big advantages in legacy environments is that, by building applications from external interfaces, it requires little or no changes to existing IT systems. Consequently, it’s an easy drop-in to app-dev and process design shops, allowing developers to boost their productivity by implementing lightweight orchestrations among built-up applications.

Going forward, Wikibon expects to see a fair amount of application development migrate to the “edges” of the business, in the form of full-fledged RPA-based developers in every line of business.

Clearly, RPA opens possibilities for enterprises who are trying to boost developer productivity under tight budget and manpower constraints. As the technology evolves, it’s very likely that RPA will becomes a more robust low-code technology. One can foresee a day when developers simply request that RPA robots automatically record any ad-hoc application workflow, revise and optimize it, and redeploy it as a repeatable, documented process across distributed microservices in a private cloud, on complex multi-clouds, or even among edge nodes on the Internet of Things.

Explore Emerging Tools That Rely Exclusively On Machine Learning For Automated Program-Code Generation

As the RPA market matures, its AI features will continue to grow in sophistication. RPA vendors are already leveraging technological advances in computer vision, sentiment analysis, topic modeling, and intent-based modeling.

In addition to ML’s central role in RPA, Wikibon seeing a growing footprint for ML-driven augmented programming as an assist in other low-code, no-code, or rapid application development scenarios. One of the more interesting recent innovations in this regard has been the use ML as the exclusive kickstarter for backward engineering of application code from mobile and Web UIs. For example, Uizard’s tool uses ML models to automatically translate raw application screenshots into code that can execute on Android, IOS, and other platforms. Other ML-augmented coding tools automatically extract executable source code from design mockups, GUI wireframes, freehand sketches, screenshots, and research papers. Still others use AI to automatically modify transplanted source code to fit the context of a different application. All of these are experimental and substantial unproven in production coding environments.

Going well beyond these limited initiatives is the more general-purpose ML-driven augmented-programming approach represented by Microsoft Research’s “DeepCoder” project. As described in this recent article and this research paper. Essentially, Deep Coder uses ML models to auto-generate program code through the following procedure:

  • Compile a supervised-learning training-data corpus of code-builds related to a particular application domain of interest.
  • Use that training data to develop machine learning models that learn the specific programmatic abstraction-layer inputs (e.g., domain-specific language specifications) most predictive of various programmatic code-build properties.
  • Search the predictive “program space” of likely code-builds, and associated properties, that might be generated from various abstraction-layer inputs consistent with some new programming challenge.
  • Assess the best-fit trade-offs (e.g., performance, efficiency, maintainability) among the most likely code-builds that address those core functional requirements.
  • Generate the best-fit code-build, from the likely program space, that is consistent with a particular abstraction-layer specification of some specific programming challenge

According to one of the paper’s authors, this approach—which they call “Inductive Program Synthesis” – targets “people who can’t or don’t want to code, but can specify what their problem is.” And they boast that their ML-augmented code-generation can solve problems of difficulty comparable to the simplest problems on programming competition websites.

However, this and other ML-augmented code-generation tools are early in their development. They’re nowhere near ready to serve as a power tool for the most complex programming challenges. For example, considering that Uizard’s offering only gets roughly 3 out of every 4 lines of code correct, this implementation of ML-driven code-gen won’t obsolete human programmers. However, as a proof of technology, it has value. If nothing else, it’s a forerunner of what are sure to be other, more accurate, and increasingly versatile code-generation tools in the future that leverage various AI techniques.

Microsoft DeepCoder’s developers hold out the possibility that in the future it might evolve to scan popular code repositories like StackOverflow or GitHub to build its corpus of code-builds that address various programming problems. DeepCoder’s developers plan to evolve it to understand the nuances of complete coding languages, and be able to recognize good code online. And as the approach evolves, it’s likely that researchers will incorporate other abstraction-layer inputs (e.g., data models, metadata, metamodels, templates, etc.) into their ML-driven code-generator.

In the next several years, we are likely to see ML-augmented coding gain adoption as a rapid application development tool. Wikibon foresees that augmented programming solution vendors will incorporate ML for the following disruptive capabilities:

  • Automatic generation of ML-optimized code-builds: Rather than “re-invent the wheel” with handcrafted code or repurposed code modules, future developers may simply check off program requirements in a high-level GUI, and then, with a single click, auto-generate the predictively best-fit code-build, as determined by embedded ML/AI, into the target runtime environment.
  • Source-code repositories for training ML-driven auto-programming: Tools for optimizing the AI models in low-code and RPA environments will also grow more versatile. Within the next several years, Wikibon predicts that developers will be able to automatically train their code-generation jobs through supervised learning on a deepening pool of field-proven code-builds managed within their low-code and RPA tools.
  • Self-learning auto-programmed bots in live applications: ML-driven RPA bots will become self-learning. This will enable them to continuously retrain themselves with live application data. In the process, they will be able to dynamically revise, optimize, and redeploy application code in production environments.
  • Auto-programmed bot self-tuning through ongoing stakeholder engagement: Tomorrow’s augmented programming bots will leverage NLP and digital assistant technology to the engage in ongoing dialogues with developers, analysts, and domain experts. This will enable those RPA agents to assess whether they’ve accurately inferred the business requirements that drive automated code-builds.

However, even as it grows more sophisticated, augmented programming is not likely to become entirely automated through ML/AI or any other means. Though data-driven algorithms will soon do the bulk of low-code auto-programming, development shops will still need expert humans in the loop.

Humans will remain an essential player to confirm that low-code, RPA, and AI bots are building the necessary code and that their deployed apps are achieving the desired outcomes.

Action Item

Organizations should adopt visual augmented-programming tooling to boost developer productivity. Explore mature “low-code” tools for general-purpose code-generation and for narrowly scoped application domains, including Web, mobile, and business process management. Make sure that the tooling integrates with your development shops’ DevOps tools for continuous integration, issue tracking, schema migration, monitoring, security, and scaling associated with auto-generated application code. Venture into RPA for UI-driven augmented programming of office productivity applications. Give preference to low-code and RPA tools that use ML and other AI approaches to speed automation of key development tasks.

Book A Briefing

Fill out the form , and our team will be in touch shortly.
Skip to content