When Automation Becomes a Security Risk
In July 2025, Jason Lemkin, founder of SaaStr, was using Replit’s AI coding agent to develop a web application. After several days of development, he placed the project under a code freeze and repeatedly instructed the agent not to make changes without his permission.
Despite these instructions, the agent executed commands that deleted a production database containing records of 1,206 executives and more than 1,196 companies. It subsequently reported that the data could not be recovered, although the records were eventually restored.
Replit publicly acknowledged the incident and announced changes to separate development and production databases.
The incident exposed a security concern that goes beyond AI making mistakes. The agent had been instructed not to make changes, but it still had the technical ability to execute commands that affected production data.
The system had relied on the agent following instructions rather than having sufficient technical restrictions and permissions to prevent destructive actions, such as database deletion.
As businesses introduce more AI-powered automation into their operations, this distinction becomes increasingly important. The more responsibility we give automated systems, the more carefully we need to consider what they can access and what they are allowed to do.
How much control are we giving automation?
Automation itself is nothing new. Businesses have relied on it for decades, from manufacturing processes to financial transactions and software deployment.
What is changing is the level of independence we are giving some of these systems.
Traditional automation generally follows predefined rules. For example, a company might automatically process purchase orders, update inventory records or generate invoices when specific conditions are met.
AI agents can go further. They can interpret instructions, select available tools and carry out tasks without every individual action being predefined.
A business might use an AI assistant to manage customer requests, update records or perform administrative operations across several applications. These capabilities can save time and reduce repetitive work, but they also introduce security concerns.
An assistant that only needs to retrieve customer information might be connected using credentials that also allow it to modify or delete records. Similarly, a development agent might receive access to a production database even though its work should be limited to development and testing.
These configurations may be convenient, especially during development, but they determine how much damage an automated system could cause if it makes a mistake or its access is compromised.
Instructions are not security controls
The Replit incident illustrates the difference between telling software what to do and controlling what it is technically allowed to do.
An AI agent can be instructed not to delete files, modify customer records or execute certain commands without approval. However, if its credentials and available tools still permit those operations, the instructions alone cannot prevent them.
The same principle applies to ordinary automation.
A workflow might be configured to update only certain customer records while using an account with administrative privileges over the entire database. It could operate correctly for years, but those additional permissions remain a security risk.
This is why access control should be enforced by the systems themselves, not left to the automation’s instructions.
If an assistant only needs to retrieve information, it should have read-only access wherever possible. If it needs to modify records, those changes should pass through an application interface that validates the request and enforces the appropriate permissions.
AI agents introduce additional challenges because they can choose how to carry out a task. Even when their objective is legitimate, the actions they select may not always be what the developer or user intended.
When connected systems create additional risks
The concern extends beyond AI agents.
Many businesses connect their CRM, accounting, customer support and other applications to exchange information and automate operations. Each integration requires some level of access to another system.
In August 2025, Google’s Threat Intelligence Group reported that attackers used compromised OAuth tokens associated with Salesloft Drift integrations to access data from connected Salesforce environments.
Unlike the Replit incident, this involved attackers misusing integration credentials rather than an AI agent making a mistake.
However, both cases demonstrate why the permissions granted to software deserve careful attention.
A company might have strong security controls around its main application but still expose sensitive information through a compromised third-party integration.
Securing individual applications is therefore not enough. Businesses also need to understand how their systems are connected, which permissions those connections require and what happens if those permissions are misused.
Building automation with security in mind
Automation offers real benefits, and avoiding it is not a practical solution for businesses that depend on efficient digital processes.
The better approach is to build security into the way automated systems operate.
That begins with least-privilege access, ensuring each agent or workflow receives only the permissions necessary for its task.
Development and production environments should also be separated. An AI coding agent used for testing should not automatically have access to live customer data or production databases.
For sensitive operations such as deleting important records, modifying user permissions or authorising financial transactions, an independent approval mechanism can provide another layer of protection.
Logging and monitoring are equally important. Businesses should be able to trace what an automated system accessed, which actions it performed and whether those actions succeeded or failed.
Finally, appropriate backups, recovery procedures and the ability to revoke access help contain damage when something goes wrong.
These are established security practices. Their importance grows as businesses give automated systems greater access to applications, infrastructure and sensitive information.
Automation should make work easier, not security harder
The Replit incident could have been much worse if the deleted data had not been recoverable. It also showed why separating development from production is more reliable than depending on an agent to follow instructions.
Before connecting an automated system to a database, CRM or financial platform, businesses should understand what it can access, what it can change and which safeguards prevent unintended actions.
At Dev@Work, this is an important consideration when designing software and integrating systems. Security should be built into the permissions, architecture and controls that determine what an automated process can do, rather than depending entirely on whether it behaves as expected.
Automation can improve productivity and reduce manual work, but it should not come at the cost of losing control over critical systems.
The goal is not only to make automation work correctly, but also to limit what can go wrong when it doesn’t.