As generative AI moves into manufacturing and semiconductor operations, the conversation is shifting from “How powerful is the model?” to a much more practical question:
Is the enterprise’s existing data and IT infrastructure actually ready to support AI in production?
Once AI begins connecting to ERP, MES, SOPs, and production data, the model itself becomes only one layer of a much larger architecture. How data is integrated, how business meaning is defined, what information AI is allowed to access, and how models and GPU resources are managed all determine whether AI can move beyond a proof of concept and into real operations.
From an engineering perspective, MICH Senior AI Engineer Wayne Huang argues that enterprise AI should not be approached simply as the deployment of a large language model. What enterprises really need is an Enterprise AI Infrastructure that can continuously support multiple AI applications over time.
The Most Time-Consuming Work Happens Before Model Inference
One of the most underestimated costs of enterprise AI is data engineering.
Imagine that a new APS application or AI Agent requires 18 data tables, while the existing MES contains 50 tables and the ERP contains another 60.
The challenge is not whether the data exists. The real challenge is identifying the correct sources, mapping schemas across systems, transforming the data into the required format, and validating the results.
In other words, the bottleneck often appears before the model even begins inference.
LowsiteOS Data Agent is designed to address this layer by helping identify source data and establish mapping and transformation pipelines. Critical transformations can remain subject to human approval through a human-in-the-loop mechanism to maintain data accuracy and accountability.
The principle is simple:
AI handles repetitive engineering work at scale, while people retain control over critical decisions and accountability.
Accessing Data Is Not the Same as Understanding It
Once data is connected, the next challenge is the Semantic Layer.
Enterprise databases are often the result of years of system development. Even if an AI model can read the schema, it does not automatically understand the business meaning behind every table and column.
Metrics such as yield, OEE, and cycle time may each involve different formulas, exclusion rules, and time ranges depending on the organization or department.
Enterprise AI therefore needs more than a Data Catalog. It requires a data foundation capable of defining Table Semantics, Column Semantics, and Metric Semantics.
This is also an important mechanism for reducing AI hallucinations.
Rather than allowing the model to guess what enterprise data means, organizations can provide the correct business context directly to the AI.
AI Must Follow the Same Identity and Access Rules as Everyone Else
As AI gains access to core enterprise data, Identity Governance becomes a fundamental part of the architecture.
If an engineer is authorized to access only Fab A, an AI query should not expose information from Fab B simply because the backend is operating through a highly privileged service account.
AI should never become a “superuser” operating outside the company’s existing access-control framework.
A more sustainable architecture is one in which AI data access inherits the user’s own identity and permissions, while maintaining complete access logs.
The rule should remain consistent:
If a user cannot access the data directly, they should not be able to access it through AI either.
This brings AI into the enterprise’s existing Zero Trust and data governance framework, rather than treating it as a separate layer with unrestricted access.
On-Premises AI Is About More Than Keeping Data Inside the Factory
On-premises AI is often discussed primarily in terms of cybersecurity and data sovereignty. From an engineering perspective, however, its broader value is control.
Enterprises gain control not only over where their data resides, but also over:
Model versions, API endpoints, permissions, logs, and GPU resources.
The LowsiteOS Model Layer uses an OpenAI-compatible endpoint, allowing upper-layer AI applications to remain decoupled from the underlying model.
This means applications such as RAG systems, MES copilots, and AI Agents do not have to remain permanently tied to a specific model. The underlying model can evolve as requirements change, while the application and existing Data Layer remain intact.
This flexibility is particularly important in an AI market where models continue to evolve rapidly.
The enterprise asset worth accumulating is not simply one specific model. It is the organization’s own data, semantics, permissions, and application architecture.
Model Management Is Becoming GPU Resource Management
Once AI is deployed on premises, another engineering challenge quickly emerges: GPU VRAM.
Most enterprises will eventually operate more than one model. Different Agents, RAG applications, vision workloads, and inference tasks may require different models.
If every model remains loaded in GPU memory at all times, hardware resources can quickly become the next bottleneck.
The Model Layer therefore also needs to manage the model lifecycle—loading and unloading models based on usage, releasing VRAM when resources are idle, and dynamically reallocating compute capacity according to workload requirements.
This marks an important shift in on-premises AI:
The challenge is no longer simply whether an enterprise can run AI models, but how efficiently it can manage limited and expensive AI computing resources.
From Copilot to Agent: Permissions Must Expand Gradually
As AI evolves from answering questions to taking action as an Agent, governance requirements become significantly more demanding.
Wayne describes Agent capabilities as a progression:
Read → Analyze → Recommend → Execute
AI can begin by reading enterprise data, analyzing information, and generating recommendations. But once an Agent is allowed to write back to ERP, MES, or other mission-critical systems, stronger authorization controls and human approval mechanisms become essential.
At the same time, enterprises need a complete audit trail covering prompts, responses, model versions, data sources, user identities, and approval records.
It is no longer enough to know simply what the AI did.
Organizations need to be able to answer:
Who used which model version, under what permissions, accessed which data, and ultimately generated which decision or action?
This level of traceability is one of the key differences between consumer generative AI and enterprise-grade AI.
From AI Projects to Enterprise AI Infrastructure
For Wayne, enterprise AI infrastructure can ultimately be summarized through four core principles:
Foundation, Semantic, Identity, and Accumulation.
Foundation establishes a reliable and reusable data layer. Semantic gives AI the business context required to interpret enterprise data correctly. Identity ensures that AI follows existing access and governance policies. Accumulation allows the data, semantics, permissions, and governance established in the first AI project to be reused by the second, third, and every subsequent Agent.
This is the fundamental difference between building isolated AI projects and establishing an AI platform.
Once the underlying infrastructure is in place, every new AI application does not need to reconnect to ERP from scratch, redefine the same fields, or rebuild another permission model.
Instead, each application can build on the foundation that already exists.
The Next Competitive Advantage Is Governable, Scalable AI
Models determine what AI is capable of doing.
But data and governance determine whether AI can actually operate inside the enterprise.
For semiconductor and manufacturing companies, the next stage of AI competition will therefore not be defined simply by who deploys an LLM first.
It will be defined by who can build an Enterprise AI Infrastructure that is governable, scalable, traceable, and designed to accumulate value over time.
