Why We Need A Data-Centric OS For AI

It is a pretty bold claim for anyone to call themselves the creator of the operating system for AI. Many have done it, but because of the architectural focus shifting away from raw compute and towards sophisticated data – something more than just storage – only one or two are going to get traction and ultimately be successful selling an AI operating system.

Jensen Huang, co-founder and chief executive officer at Nvidia, started calling its Dynamo AI inference stack an AI operating system during his keynote address – technically, he called it the operating system for the AI factory – back in March, and we sort of stashed that idea away for further processing. But then last week, Vast Data, the upstart flash storage maker that is layering all kinds of software on top of it to turn it into what it has called up to this point a “data platform” proclaimed that it had, in fact, created the Vast AI Operating System. And this got us to thinking some more.

What Nvidia and Vast Data are talking about is distinct from, but perhaps ultimately intertwined with, what we also expect to see, which is an operating system powered by AI. We are not talking about this right now. Just to be clear.

What we are talking about is a stack of software running on what we presume will be commodity hardware that allows companies to run pre-existing AI applications or create ones themselves, or a mix of the two.

Compute As The Center Of The Universe

To say that the “data center” has, despite its name, been obsessed with compute for the past six decades is an understatement.

The original IBM System/360 “central processing units” from 1964, who architecture has pretty much defined computing since that time, were housed in a “main frame” and had a bunch of peripherals for displaying data on screens, storing and reading it on tape drives and then disk platters, and printing it our in reports and other kinds of documents like an invoice or a bill of materials. These mainframes were put in “glass houses” so executives could show off their technical prowess.

These CPUs in the mainframe were literally at the heart of the system, and it is where data was manipulated and how applications interfaced with the outside world. The whole shebang was called “data processing” long before it was called “information technology,” and that was perhaps the best term because, like Einstein’s concept of unified space-time – matter tells space how to warp, and space tells matter how to move – both “data” and “processing” were of equal status in the beginning.

But, storage has historically been a whole lot easier than compute, and less expensive, too. But in recent decades, as more kinds of data became available and that variety was necessary to drive the business – more complex relational and NoSQL databases, Hadoop and other datastores for vast amounts of unstructured data – there are plenty of companies that hoovered up massive troves of data and turned it first into “big data” to drive our experiences on the Internet and then to teach the Internet to “think” and demonstrate its “thoughts” through generative AI algorithms in all of the media that human beings create and process themselves except touch, smell, and taste. Still image (sight), text (a kind of sight with inner hearing), sound, and video (moving images and usually sound). Smell and taste are closely related, so maybe there are only four human senses, really. And eventually, we figure someone will do a better job of having humanoid robots experience and deliver these senses as well.

This got us to wondering what would happen if, from the beginning, compute was cheaper and easier and data was the center of the process, as Vast Data is implying it should be with its OS for AI.

Before we get into that, what precisely do we mean by operating system? We have all been around them for so long, and their installation and updating is so automated, that they are a bit invisible like the air we breathe.

Way back in the dawn of time – the late 1990s – we referred to “Linux” as an “operating system,” and received an upbraiding by none other than Richard Stallman, the patron saint of the GNU Project, creator of many system software components and compilers that could run on various Unix kernels. He explained that Linux was but an operating system kernel, and it was GNU and other pieces of code that turned Linux into an operating system. We listened to this argument and agreed, and continued to call Linux an operating system because that is how the world talked about it. In any event, Linux is the last mainstream compute-centric operating system to come to market, and probably will be the last one created in our lifetimes. Linux will live on in many forms for all kinds of devices – Android for laptops and smartphones, RHEL and SUSE for servers, and so on.

So, you start with a kernel, and you wrap device drivers and other services (storage, networking, and so on) around it to create an operating system. In the datacenter, this operating runs on server, storage, and network devices. A server or a collection of servers with distributed computing software and hardware to accelerate it (NICs and DPUs) plus switching and storage (devices to move data between servers and to store data for servers to use for processing) is a base system.

A full system includes systems software – databases, middleware, and other higher level programs, including programming language runtimes – that allows for it to run applications, which are created by customers themselves or by third party experts or a mix of the two. Everything below the applications we call a platform, and as the name of this publication implies, we are on the hunt for them all the time.

Not everyone hews to our parfait of hardware and software, but the basic gist is there is a kernel at the bottom, a whole bunch of layers in the middle, and applications on top. Some call those middle layers and operating system, some call it a platform. Call it what you may, but Vast Data says it has created one for AI.

So have a lot of others. Let’s roll off a few.

• Nvidia with Dynamo, for starters, as we mentioned above.
• Google DeepMind has created something called AG2 that it bills as an “open source agent OS,” but this looks like a higher level workflow for creating agents and managing them without having a full stack approach for managing data and the training of models as well as inference.
• AI Dynamics, which has been at this since 2015 (a year before Vast Data was founded), has a platform called NeoPulse that its investors claim is “the operating system for AI.”
• XLsoftware, one of the commercializers of the Anaconda data analytics platform, calls the very wide and deep stack of systems software for AI and data analytics that it sells “the operating system for AI.”
• Way back in 2017, when the GenAI wave was five years into the future, Algorithmia was explaining how to build an operating system for AI. It appears to be part of DataRobot now.
• Harkening back to the early days of the PC revolution, Eliza:OS says that it is “the operating system for AI agents.” Like AG2, Eliza:OS looks more like a video game than a serious attempt at enterprise software.
• Letta has “the operating system for AI agents,” which helps you create stateful agents and a development environment to see how that agent is reasoning.

We could go on and on.

This is a natural enough way to talk about creating an integrated, self-contained, easy to use AI factory, to use Nvidia’s term for it.

But we think that Vast Data is different in that it is focused on the data first, and the many ways it needs to be housed, formatted, accessed, manipulated, and distributed to AI agents for it to be useful. Old fashioned compute architectures take a top down approach, creating complexity to get many sources of data to feed into a single compute paradigm. What Vast Data is doing with its OS for AI is masking the underlying complexities of all kinds of data with its flash-tuned element store and disaggregated shared everything (DASE) storage architecture to scale across large amounts of data and present it in just about any form an AI application might desire. This is a bottom up approach, and getting the data right is a true differentiation. Anyone can do compute. Well, anyone named Nvidia and AMD, at least, and probably HiSilicon.

Perhaps The Only Way To Make An OS For AI

Given all of this talk about operating systems and platforms, we asked Renen Hallak, one of the company’s co-founders and its chief executive officer, why the Vast Data Platform – a kind of flash storage that was architected to be file, block, or object storage and more recently a relational and vector database and a virtual Kafka streaming cluster – was widening further out to be an operating system for AI. And, importantly, was not being called an AI platform.

“I think the idea is to emphasize the fact that it’s all of the software infrastructure parts,” Hallak tells The Next Platform. “It’s not just the storage solution, it’s not just the database – it’s also the runtime, it’s also the security. But an OS for AI is also the way that you manage and observe the AI, so when you want to know what Agent A talked to Agent B about three years ago, you can go into the Kafka stream that’s persistent and audit it and know exactly what happened. The OS for AI also about enabling the communication between agents within this stack as well as communication between people and agents. It’s a communication between agents and the natural world. But more importantly, it’s not just storage, and it’s not just database. Storage is the unstructured piece – all the information as it flows in through sensors. It’s also all of the memories that accumulate over time, the historical data, databases making meaning of it and generating insight from it, understanding what’s in all of these pictures and sounds and words that are flowing in. But now we need everything else. We need the ability to discover and identify new agents, to manage the ability for them to interact with each other and to limit what it is that they can and cannot do depending on who they’re talking to. All of those actions should be orchestrated and scheduled, and be part of this operating system for AI.”

And you thought Vast Data was just about banning the disk from the datacenter, or making flash faster and cheaper, or making NFS cool again and obviating the need for other file systems, or blurring the lines between storage and database.

Anyway, we think that everyone always gets pulled up the value chain because that is where the profits are. Which is why few modern storage vendors make their own hardware, and why Vast Data stopped making hardware appliances years ago. There are some storage makers that do still make hardware. But most are just running storage on standard hardware and then services on compute within the clustered storage. Vast Data has various hardware partnerships to get iron for its software to run on, which includes D-Nodes for storing data and C-Nodes for doing various kinds of compute, all hooked into a single distributed but disaggregated namespace that spreads the data and the work necessary to manage it over a large number of devices to boost throughput and lower latency. The C-Nodes also support Kubernetes containers to run applications right next to the data on the DASE cluster.

Hallak says that Vast Data now has a bunch of people in research and development creating agents and stringing them together to make workflows – what we might call an application. One of these agentic apps was showed off during Huang’s keynote address at Computex a few weeks ago. Vast Data does not want to be an application supplier, but it has to create some applications to demonstrate the power and ease of its “operating system for AI” and to understand the issues that customers will run up against.

As far as we know, this OS for AI will be bundled as part of the complete Vast Data stack, but there is every reason to believe that the company will eventually allow for different blocks of the OS block diagram to be replaced with compatible but different blocks, in much the same way that Linux distros allowed multiple file systems and databases within their distros. Being a relatively young and small company (six years and close to 1,000 employees, with a $10 billion valuation) means Vast Data has to limit the support matrix for a while yet.

We would remind the company as it moves ahead with this OS for AI that there is always tension between “best of breed” components that customers want to choose from and the integrated approach that Vast Data is taking as it builds its stack. At a certain point the OS for AI will have its support matrix widened, just like Linux. Moreover, we would point out that integrated does not mean included by default and invoiced by default. You can integrate a platform or an operating system without charging for every component or even bundling every component. An integrated platform or operating system should allow for a la carte and not be, as the French say, table d’hôte, which is literally “the host’s table” and roughly synonymous with prix fixe, meaning a fixed price for a fixed menu. The more flexibility, the better.

But clearly, Vast Data wants to get greater wallet share of the many hardware appliances and software stacks that enterprise customers have today as they do analytics and AI.

“Over time, the budget for what we do is shifting more towards software, less towards hardware,” Hallak explains. “And the reason for that is that we are providing more and more pieces of the stack, and so we are displacing more and more of the older software systems. The other effect that we are seeing is the hardware keeps going down in price per capacity, but the amount of the hardware capacity is growing so fast that the average deal size, including the software, is growing even faster because we are taking wallet share away from more and more parts of the stack. It’s no longer just displacing the storage system or the database or the orchestration layer, or the streaming system for Kafka. It’s now all of these. And so our software grows proportionately, and the hardware shrinks proportionately.”

One last thing. Operating systems come out of weird places. The C programming language and its first application, the Unix operating system, happened because AT&T needed a more efficient way for creating our phone bills. Windows, and therefore Windows Server, came from some punk kid in Seattle who created a BASIC compiler and was in the right place at the right time when IBM was too lazy to create an OS for its own PC line. So why can’t a storage and data management vendor create an operating system for AI? It makes as much sense as anything else we have seen. And clearly, we need one.