Divide and Conquer
Language models are abstracting out the traditional interface—programming—required to apply software engineering paradigms to real-world problems.
One of the classic software engineering design patterns is the divide-and-conquer approach.
You take a large arbitrary problem, break it down into more manageable component pieces (either mentally or computationally), and solve for each piece of the problem one by one. Then, you integrate each solved part of the problem until you have a collection that is able to solve the larger more abstract problem.
Divide and conquer is about as basic a methodology as you can get in software development for tackling real-world problems. Large language models are now showing they’re potentially a kind of “interface” for more arbitrary problems—and opening the door to bringing that paradigm (and potentially more down the line) one abstraction layer up.
Two emerging areas are really interesting when tackling more general-purpose problems in a performant and effective manner: limited-purpose agent networks, and mixture of experts approaches.
The former allows a company to solve a complex, but narrow, task. But with an agent attacking each individual part of a problem and providing the ability to communicate amongst themselves, you start to get a network that can break down an arbitrarily large problem into smaller ones that sum up to automating a full workflow. The task could be even as narrow as adding a scheduled time from an email onto a calendar.
The latter, based on a big report out of SemiAnalysis, is also employed by OpenAI with GPT-4 and allows a company to take a more arbitrarily difficult problem and delegate the components of the problem to a set of specialized models that are able to resolve them in an efficient and quick matter. In the extreme case of GPT-4, it’s a completely open-ended language interface that answers as much as it possibly can as accurately as it can.
What’s fun about all this is divide-and-conquer is one of many design paradigms in software engineering. But language models are starting to show us some interesting signs of being able to throw many of those principles to tackle real-world problems into a bag and take the elevator one floor up on the abstraction layer.
To be clear, deploying these techniques is about ten (to one thousand) steps ahead of where enterprises currently are in their adoption cycle. We’re generally at the “how can I use an LLM to fit into my workflow in the first place” stage for a lot of companies.
But, hey, that’s part of the fun of all this development: it all starts with a toy until it gets bigger and bigger and becomes a core use case.