Network automation projects rarely fail. They stall, quietly, somewhere around month six.
Nobody cancels anything. The platform stays up, the licenses get renewed, and there is still a line for it in the roadmap. But new workflows stop shipping, the people who were excited about it drift to other work, and changes go back to being typed into devices by hand. We have been asked to look at projects like this on Cisco NSO, Cisco Crosswork, NetBrain, Forward Networks, and plain Python. The platform changes. The patterns don't. Each one below comes with the evidence that reveals it, because a stall is a measurable condition, not a feeling.
1. The first workflow was too big
The first thing the team tried to automate was the hardest thing they do: a full customer turn-up, every platform, every variant, every option anyone has ever ordered. A year later it still isn't done, and there is nothing in production to show for it.
The evidence is the ratio of model to instances. On NSO, count the leaves in the service YANG and count the service instances in production:
admin@ncs# show running-config services l3vpn | count
Count: 0 lines
admin@ncs# show packages package l3vpn package-version
packages package l3vpn
package-version 0.0.14Fourteen versions of a service model with zero instances is a project that has been designing instead of shipping.
The fix is to ship one narrow workflow within 30 days. Pick something frequent, low-risk, and boring: an interface description, a VLAN on an access port, a pre-change check that runs before every maintenance window. A model with four leaves and two hundred instances teaches a team more than one with forty leaves and none.
2. A year on the source of truth, and nothing on the network
The project started with the inventory, because every vendor and every consultancy said it should. Eighteen months in, NetBox or Nautobot is beautifully modeled, half populated, and no change has ever flowed from it to a router.
The evidence is in the webhook and job logs. A source of truth that drives automation emits events that something consumes:
$ curl -s -H "Authorization: Token $NB_TOKEN" \
"https://netbox.example.net/api/extras/webhooks/" | jq '.count'
0Zero webhooks after a year of modeling means the data has no consumer. It is a database, not a source of truth.
A source of truth is necessary. It is not a deliverable. The fix is to build it as the workflows demand it, not before. Pick the first workflow, model only the data that workflow needs, wire the webhook or the pipeline that reads it, and let the inventory grow one workflow at a time.
3. The network didn't match the data
Half the devices show drift between what the platform thinks is there and what is actually configured. Nobody wants to reconcile, because nobody is sure what the sync will pull in or overwrite. So the gap widens every week, and every workflow has to work around it.
The evidence is a single command, repeated weekly and plotted:
admin@ncs# devices check-sync | include out-of-sync | count
Count: 61 lines
admin@ncs# show devices list | count
Count: 118 linesMore than half the estate out of sync is not a data problem. It is a process problem: out-of-band changes are normal and nothing notices them.
The fix is to make drift visible and boring. Run the comparison on a schedule, publish the report, and work through the devices one at a time with compare-config before any sync-from. Then agree on a policy for out-of-band changes, so the network doesn't drift again the week after the reconciliation finishes. There is a full procedure for NSO in a separate post.
4. No path from a laptop to production
The workflow works on the developer's machine. Getting it into production means copying files onto a server, editing them in place, and hoping. There is no history, no review, and no way to know whether a change breaks something until it does.
The evidence is on the server itself:
$ ls -la /var/opt/ncs/packages/
drwxr-xr-x vlan
drwxr-xr-x vlan.bak
drwxr-xr-x vlan.bak2
drwxr-xr-x vlan-working
$ cd /var/opt/ncs/packages/vlan && git log --oneline | head -1
fatal: not a git repositoryFour copies of a package and no repository is a project that cannot roll back.
The fix is a pipeline, even a small one. Git as the source of truth for the code, netsim or a lab to run against, and a dry-run diff that somebody reads before anything is committed:
admin@ncs(config)# vlan access-1042 vlan-id 1042 device pe2-den
admin@ncs(config-vlan-access-1042)# commit dry-run outformat native
native {
device {
name pe2-den
data interface GigabitEthernet0/0/1.1042
description vlan-1042
encapsulation dot1q 1042
!
}
}If the diff surprises the reviewer, it would have surprised everyone in production too. The pipeline post walks through building this on GitLab.
5. The process was never written down
The automation encodes a process. Nobody wrote that process down, so the developer guessed. The workflow asks for inputs operators don't have, skips the approval the change board requires, and doesn't know about the one thing the senior engineer always checks. Operators work around it, and then they stop using it.
The evidence is in the ticketing system. Pull the last ninety days of network changes and count how many reference the platform at all:
$ curl -s -u "$SN_USER:$SN_PASS" \
"https://example.service-now.com/api/now/table/change_request?sysparm_query=category=network^opened_at>javascript:gs.daysAgo(90)&sysparm_fields=number,short_description" \
| jq '[.result[] | select(.short_description | test("nso|crosswork|automation"; "i"))] | length'
3Three of several hundred changes touched the platform. The other several hundred went around it, and each one is a process the automation didn't model.
The fix is to map the process before building anything. Sit with the people who request, approve, and make the change, and write down what actually happens, including the spreadsheet and the chat message. Twenty minutes walking an operator through the inputs saves weeks of rework later.
6. The one person who built it left
One engineer built most of it, and that engineer moved on. What is left works, mostly, but nobody understands it well enough to change it. Every new request is a risk, so the answer is always no.
The evidence is the commit history, when there is one:
$ git shortlog -sn --since="2 years ago" packages/
214 jsmith
3 kpatelThe fix is to document what exists, refactor the critical paths first, and pair with the team's own staff while doing it. The work isn't finished until there is a handoff runbook the team has actually used without the author in the room.
7. Nothing consumes it
The platform works. Requests still arrive as tickets, an engineer still reads each one and types the change, and the platform is one more tool they could use if they had time.
The evidence is the audit log. A platform that is consumed shows API calls from something other than a person at a CLI:
$ grep -c 'restconf' /var/log/ncs/audit.log
0
$ grep -c 'ncs_cli' /var/log/ncs/audit.log
1872The fix is to connect the platform to the request path: a form, an API call from the ticketing system, a button on the ticket itself. On NSO that is one RESTCONF call per change, which the ticketing system can make directly:
$ curl -s -u "$NSO_USER:$NSO_PASS" -X POST \
-H "Content-Type: application/yang-data+json" \
-d '{"vlan:vlan":[{"name":"CHG0048211","vlan-id":1042,"device":["pe2-den"]}]}' \
"https://nso.example.net/restconf/data"Make it the normal way changes happen, and track how many changes go through it so adoption can be measured instead of assumed.
A quick self-check
If three of these sound familiar, the project is stalled, not slow. None of them are technical dead ends. Each one has a fix that fits into a few weeks of focused work, and none of the fixes require changing platforms.
If a project got stuck in a way that isn't on this list, or got moving again, tell us about it in the comments.