Only 26 percent of enterprises say their AI governance keeps pace with deployment, even though 55 percent are actively running AI in production, according to the 2026 Enterprise AI Trends Study from Smarsh and FTI Consulting. Large language models are the clearest example of that gap. Providers ship new model versions on their own schedule, teams adjust prompts and retrieval sources weekly, and few organizations treat any of it as a formal change.
That gap matters more once a system is classified as high risk or feeds a regulated decision. A model swap that quietly changes an output distribution, a prompt edit that removes a safety instruction, or a new tool connected to an agent are all changes to the AI system, whether or not anyone logged them.
This piece lays out what counts as an LLM change, how to decide when a change needs formal review, and how to build a change log that survives an audit, aligned to ISO/IEC 42001, the EU AI Act, and NIST AI RMF.
What Counts as a Change in an LLM System
Most teams think of a change as swapping one model for another. That is the least common type in practice. An LLM-based system changes any time one of its dependencies shifts: the base model version, the system prompt, the retrieval index it queries, the tools or plugins it can call, the fine-tuning data, or the guardrail configuration sitting in front of it.
Each of these can alter behavior even when the underlying model stays fixed. A retrieval index refresh can introduce stale or incorrect source documents into answers. A new tool grant can let an agent take actions it previously could not. Treating only model-version swaps as reportable changes leaves the majority of real behavioral drift outside the review process entirely.
Common categories of LLM system change
| Change type | Example | Typical trigger for review |
| Base model version | Provider deprecates GPT-4-class model, migrates to a newer version | Output format or reasoning behavior changes materially |
| System prompt or instructions | Safety clause removed or rewritten during a prompt refactor | Any edit touching risk, safety, or scope-limiting language |
| Fine-tuning or adapter update | New fine-tune trained on updated support ticket data | New training data introduces domain, population, or bias shift |
| Retrieval index or knowledge base | RAG source documents refreshed or a new data source added | New source affects accuracy for a regulated use case |
| Tool or plugin access | Agent granted a new API, file system, or payment tool | New capability expands the agent’s possible actions |
| Guardrail or filter configuration | Content filter threshold loosened for latency | Threshold change affects a control tied to a risk assessment |
How ISO 42001, the EU AI Act and NIST AI RMF Define a Reportable Change
Each framework uses different language for the same underlying idea: changes to an AI system need to be planned, evaluated, and documented before they reach production. None of the three top-ranking treatments of LLM change management line up these requirements clause by clause, which leaves teams cross-referencing three separate documents to answer one question: does this change need sign-off?
ISO/IEC 42001 addresses this directly in Clause 6.3, Planning of Changes, which requires changes to the AI management system to be carried out in a planned, controlled way that preserves the integrity of the system. It does not set a numeric threshold. Instead, it expects the organization to define its own criteria for what counts as significant.
The EU AI Act is more specific for high-risk systems. Article 3(23) defines a substantial modification as a change not foreseen in the original conformity assessment that affects compliance with Chapter III requirements or changes the system’s intended purpose. Article 43(4) then requires a new conformity assessment whenever that threshold is crossed, with one carve-out: changes that were predetermined and documented at the time of the original assessment, common for systems designed to keep learning after deployment.
NIST AI RMF folds change management into the Manage function. MANAGE 4.1 calls for post-deployment monitoring and change management processes to be in place, while GOVERN 1.7 covers the policy layer: decommissioning and phase-out procedures that keep a retired or superseded model from lingering in production.
Clause-level mapping: LLM change management across ISO 42001, EU AI Act, and NIST AI RMF
| Governance activity | ISO/IEC 42001 | EU AI Act | NIST AI RMF |
| Plan and control changes before release | Clause 6.3 Planning of Changes | Article 43(4): new conformity assessment on substantial modification | MANAGE 4.1: change management processes in place |
| Define what makes a change material | Clause 6.3 (organization sets criteria) | Article 3(23): substantial modification definition | MAP 5.1: likelihood and magnitude of impact documented |
| Pre-approve learning-system changes | Clause 8.3 AI risk treatment | Article 43(4) predetermined-change carve-out | MANAGE 2.3: procedures for third-party or updated components |
| Monitor after deployment | Clause 9.1 Monitoring and measurement | Article 61 post-market monitoring (high-risk systems) | MANAGE 4.1: post-deployment monitoring |
| Retire or roll back a version | Clause 8.1 Operational planning and control | Recital 128 context on substantial modification scope | GOVERN 1.7: decommissioning and phase-out procedures |
Govern365.ai’s AI model registry keeps this mapping live rather than static: each logged model or prompt change is automatically flagged against the ISO 42001, EU AI Act, and NIST AI RMF clauses it touches, so a compliance officer does not have to rebuild this table by hand for every audit cycle.
Deciding Whether a Change Needs Formal Review
Not every change deserves the same level of scrutiny. A typo fix in a system prompt is not the same event as swapping the underlying model. The practical solution is a materiality test applied at the moment a change is proposed, not after it ships.
The EU AI Act’s substantial modification test offers a workable structure even for organizations outside its scope: ask whether the change affects compliance with a documented requirement, and whether it changes the system’s intended purpose. If either answer is yes, the change is material and needs a review gate before release.
Three questions, asked in order, catch most of what matters. First, does the change alter the system’s intended use or the population it serves. Second, does it touch a control tied to an existing risk assessment, such as a content filter, a scope limitation, or an escalation rule. Third, does it change the data the model was trained or grounded on in a way that could shift accuracy or bias for a protected group. A yes to any of these routes the change to formal review rather than a routine deploy.
Building an LLM Change Log That Survives an Audit
A version number in a deployment script is not an audit trail. Auditors and regulators reviewing an AI management system expect to see who proposed a change, what risk assessment it triggered, who approved it, and when it went live, in one place.
An effective LLM change log captures, at minimum, the system and version affected, the change category from the taxonomy above, the materiality determination and who made it, links to any updated risk assessment or impact assessment, the approver and approval date, and the rollback plan if the change needs to be reversed. Spreadsheet-based logs work for a handful of systems. They break down once an organization is running more than a few LLM-backed applications, because nobody remembers to update a row that lives outside the deployment pipeline.
A change log tied directly to the model registry, rather than maintained separately, closes that gap: every deploy event writes its own log entry instead of relying on someone to remember.
Govern365.ai‘s audit evidence management module timestamps each change log entry and attaches the associated risk assessment automatically, so the evidence an auditor asks for during a Clause 9 internal audit already exists rather than needing to be reconstructed.
Model changes are not always controlled internally, particularly when an organization relies on a third-party provider. This makes foundation model vendor risk an important part of the change review process.
The Review Workflow: Roles, Gates and Sign-off
Change management fails most often at the handoff between the team that wants to ship a change and the team responsible for knowing whether it is allowed to. A clear workflow with named roles closes that gap without slowing every deploy down.
A five-step review workflow
- Model owner proposes the change and completes the materiality test, documenting the answer to each of the three questions above.
- If the change is non-material, it is logged and deployed on the normal release cadence, no further approval required.
- If the change is material, the risk or compliance owner reviews the existing risk assessment and determines whether it needs to be updated before release.
- A designated approver, typically a governance lead or AI risk committee for higher-risk systems, signs off in writing before deployment.
- The change log is updated with the outcome, and post-deployment monitoring is scheduled for the first review window after release.
The role split matters more than the number of steps. Model owners understand the technical change but rarely have full visibility into which risk assessments it touches. Compliance and risk owners have that visibility but should not be the ones deciding whether a prompt edit is safe to ship. Keeping the technical proposal and the compliance sign-off as separate roles, even in a small team where one person wears both hats at different times of day, keeps the review honest.
Vendor Model Updates You Do Not Control
Most enterprise LLM deployments sit on top of a third-party foundation model. That creates a category of change nobody inside the organization initiates: the provider deprecates a model version, updates weights behind an existing API endpoint, or changes default safety behavior without a version bump.
Under the EU AI Act, this does not remove the deployer’s obligations. A company that licenses a high-risk AI system without substantially modifying it typically remains a deployer, but a deployer that does substantially modify a licensed system, or puts it on the market under its own name, can be reclassified as a provider and inherit the full provider compliance burden, according to Holland & Knight’s April 2026 analysis of the Act’s compliance deadlines. Deployers should track their own configuration changes on top of a vendor model just as carefully as they track the vendor’s own updates.
Practically, this means subscribing to provider changelogs and deprecation notices for every foundation model in production, treating a silent weight update behind a stable API version as a change event worth logging even though no code changed on the deployer’s side, and building a fallback plan for provider-driven deprecations with a hard cutoff date, since those dates are set by the vendor, not the deployer’s release calendar.
Post-Update Monitoring and Rollback
A review gate before deployment only catches what reviewers anticipated. NIST AI RMF’s MANAGE 4.1 subcategory treats post-deployment monitoring as a distinct, ongoing control, not a one-time check performed at launch.
The monitoring window immediately after an LLM update deserves more scrutiny than steady-state operation, because this is when unanticipated behavior surfaces. A practical baseline includes comparing output quality or accuracy metrics against the pre-change baseline for at least the first one to two weeks, watching for a spike in escalations, overrides, or user complaints tied to the affected system, and confirming the rollback path actually works, not just that it exists on paper.
Rollback plans are frequently written and never tested. A rollback that requires re-deploying a decommissioned model version, restoring a prior prompt from version control, and reverting a retrieval index in the correct order is a multi-step process, and the first time to discover a missing step should not be during an incident.
LLM Change Management Checklist
Use this checklist at the point a change is proposed, not after it ships.
LLM change management checklist
| # | Action | Framework anchor |
| 1 | Log the change category (model, prompt, fine-tune, retrieval, tool access, guardrail). | ISO 42001 Clause 6.3 |
| 2 | Run the three-question materiality test and document the outcome. | EU AI Act Art. 3(23) |
| 3 | Update the relevant risk or impact assessment if the change is material. | NIST AI RMF MAP 5.1 |
| 4 | Route material changes to a named approver before deployment. | ISO 42001 Clause 6.3 |
| 5 | Confirm whether the change qualifies for a predetermined-change carve-out. | EU AI Act Art. 43(4) |
| 6 | Record approver, approval date, and rollback plan in the change log. | NIST AI RMF MANAGE 4.1 |
| 7 | Schedule post-deployment monitoring for the first review window. | NIST AI RMF MANAGE 4.1 |
| 8 | Track vendor changelogs for every third-party foundation model in production. | EU AI Act deployer obligations |
| 9 | Test the rollback path, not just document it. | ISO 42001 Clause 8.1 |
| 10 | Update the AI system inventory entry with the new version and effective date. | NIST AI RMF GOVERN 1.7 |
Before approving a significant model or system change, teams should review LLM evaluation evidence to confirm that the updated system still meets its performance, safety and governance requirements.
Frequently Asked Questions
Does every prompt edit need formal review?
No. Minor edits that do not touch risk, safety, or scope-limiting language can follow a normal release process. Any prompt change that alters safety instructions, scope limitations, or output constraints should go through the materiality test before deployment.
Is a provider’s silent model update behind a stable API version a reportable change?
Yes. If the underlying model weights change, even under an unchanged API version, the system’s behavior can shift. Log it as a change event and run the same materiality test used for internally initiated updates.
What is a substantial modification under the EU AI Act?
Article 3(23) defines it as a change to an AI system after deployment that was not foreseen in the original conformity assessment and that affects compliance with Chapter III requirements or changes the system’s intended purpose. High-risk systems that cross this threshold need a new conformity assessment.
How often should an LLM system inventory be updated for change tracking?
Update the inventory entry at the moment a version changes, not on a fixed schedule. NIST AI RMF’s Govern function treats the inventory as a living record tied to deployment events, so a quarterly refresh cycle leaves gaps between updates.
Who should approve a material LLM change?
A designated approver separate from the person proposing the change, typically a governance lead or AI risk committee for higher-risk systems. Keeping proposal and approval as distinct roles prevents the person closest to the technical change from also being the sole compliance check.
Does ISO 42001 specify a numeric threshold for what counts as a significant change?
No. Clause 6.3 requires changes to be planned and controlled but leaves the organization to define its own materiality criteria. Most organizations pair Clause 6.3 with a documented decision framework, similar to the EU AI Act’s substantial modification test, to make the threshold consistent and auditable.
What belongs in an LLM change log?
At minimum: the system and version affected, the change category, the materiality determination, links to any updated risk assessment, the approver and approval date, and the rollback plan. A log that lives inside the deployment pipeline stays current more reliably than one maintained separately.
Conclusion
The organizations closing the governance gap that Smarsh and FTI Consulting found in 2026 are not the ones blocking every model update with a committee review. They are the ones with a fast, consistent test for deciding which changes need one.
Start with the change taxonomy: know every way your LLM systems can change, not just model versions. Apply the three-question materiality test at the point of proposal, and keep the change log inside the deployment path so evidence exists before an auditor asks for it.
Govern365.ai, by the Global AI Certification Council, brings the model registry, risk assessments, and change log into one system so a material update is never more than a few clicks from its audit trail. Start your 14-day free trial to see how it maps to ISO 42001, the EU AI Act and NIST AI RMF out of the box.
