Scheduled Tasks

Overview

Scheduled tasks automate recurring network operations so that critical maintenance, monitoring, and deployment activities happen reliably without manual intervention. Every task runs on a cron-based schedule with timezone support, configurable retry logic, and full execution history.

NetStacks supports five task types:

How It Works

The task scheduler is built on a PostgreSQL-backed job queue. When you create a scheduled task, the Controller registers a job with the scheduler, which evaluates cron expressions to determine the next run time. The scheduler supports timezone-aware scheduling — all cron expressions are stored alongside a timezone identifier and converted to UTC for execution.

Execution Lifecycle

Every task follows this lifecycle:

1. Scheduled — The scheduler calculates next_run_at from the cron expression and timezone
2. Pending — The job enters the queue when next_run_at is reached
3. Running — The executor picks up the job and begins execution against target devices
4. Completed or Failed — The task finishes successfully or fails after exhausting retries

Retry Logic

Each task has three retry-related settings pulled from the task configuration:

• max_retries — Maximum number of retry attempts (default: 3)
• retry_delay_seconds — Delay between retry attempts in seconds
• timeout_seconds — Maximum time a single execution attempt can run before being killed

Task Parameters

Each task type accepts type-specific parameters stored as a JSON payload. For example, a Backup task requires a target_type, target_id, and destination, while an AgentTask only requires a prompt string. The Controller validates parameters against the expected schema before saving.

Creating Tasks

Follow these steps to create a new scheduled task from the NetStacks UI.

Step 1: Navigate to Scheduled Tasks

Open Automation → Scheduled Tasks in the main navigation.

Step 2: Click New Task

Click the New Task button in the top-right corner of the task list.

Step 3: Select Task Type

Choose from Deployment, Backup, HealthCheck, CustomScript, or AgentTask. The form fields update based on your selection.

Step 4: Configure the Schedule

Enter a cron expression and select a timezone. Use the Cron Expressions reference for syntax help. The UI previews the next 5 scheduled runs so you can verify the schedule is correct.

Step 5: Set Task Parameters

Configure the type-specific fields:

• Deployment — Select the stack instance to deploy
• Backup — Choose target type (device or group), select the target, and specify a backup destination
• HealthCheck — Select a credential and choose which checks to run (ping, SSH, SNMP)
• CustomScript — Paste or select a script and optionally bind a credential for device access
• AgentTask — Write the natural-language prompt for the AI agent to execute

Step 6: Configure Retry Options

Set max retries, retry delay (seconds), and timeout (seconds). Sensible defaults are pre-filled but can be adjusted for tasks that need longer timeouts (e.g., backups across many devices).

Step 7: Enable and Save

Toggle the Enabled switch (tasks are enabled by default) and click Save. The task appears in the list and the scheduler calculates the first next_run_at timestamp.

Code Examples

Nightly Configuration Backup

Back up running configurations from all core routers every night at 2:00 AM Eastern:

{
  "name": "Nightly Core Router Backup",
  "task_type": "backup",
  "cron_expression": "0 2 * * *",
  "timezone": "America/New_York",
  "enabled": true,
  "max_retries": 3,
  "retry_delay_seconds": 60,
  "timeout_seconds": 600,
  "parameters": {
    "type": "backup",
    "target_type": "device_group",
    "target_id": "a1b2c3d4-5678-9abc-def0-123456789abc",
    "destination": "config-snapshots/core-routers"
  }
}

Edge Switch Health Check Every 15 Minutes

Monitor connectivity to all branch edge switches with SSH and ping checks:

{
  "name": "Edge Switch Health Monitor",
  "task_type": "health_check",
  "cron_expression": "*/15 * * * *",
  "timezone": "UTC",
  "enabled": true,
  "max_retries": 2,
  "retry_delay_seconds": 30,
  "timeout_seconds": 120,
  "parameters": {
    "type": "health_check",
    "credential_id": "b2c3d4e5-6789-abcd-ef01-234567890bcd",
    "checks": ["ping", "ssh", "snmp_sysname"]
  }
}

Weekly VLAN Deployment

Push updated VLAN configurations to access-layer switches every Sunday at midnight Pacific:

{
  "name": "Weekly Access Switch VLAN Push",
  "task_type": "deployment",
  "cron_expression": "0 0 * * 0",
  "timezone": "America/Los_Angeles",
  "enabled": true,
  "max_retries": 1,
  "retry_delay_seconds": 120,
  "timeout_seconds": 900,
  "parameters": {
    "type": "deployment",
    "stack_instance_id": "c3d4e5f6-7890-bcde-f012-345678901cde"
  }
}

AI Agent Morning Summary

Have the NOC agent generate a network health summary every weekday at 7:00 AM:

{
  "name": "Morning Network Health Summary",
  "task_type": "agent_task",
  "cron_expression": "0 7 * * 1-5",
  "timezone": "America/Chicago",
  "enabled": true,
  "max_retries": 2,
  "retry_delay_seconds": 60,
  "timeout_seconds": 300,
  "parameters": {
    "type": "agent_task",
    "prompt": "Summarize the current health of all BGP peers, report any interfaces with errors above threshold, and flag any devices that missed their last backup window."
  }
}

Questions & Answers

Q: What task types are available in NetStacks?

A: NetStacks supports five task types: Deployment (push stack instances to devices), Backup (capture device configurations), HealthCheck (verify connectivity and device status), CustomScript (run arbitrary automation scripts), and AgentTask (execute AI agent prompts). Each type accepts different parameters tailored to its operation.

Q: How do I schedule a task to run at a specific time?

A: Every scheduled task uses a cron expression paired with a timezone. For example, to run a backup at 2:00 AM Eastern every day, set the cron expression to 0 2 * * * and the timezone to America/New_York. The scheduler converts this to UTC internally and calculates the next run time. See the Cron Expressions guide for syntax details.

Q: What happens if a scheduled task fails?

A: When a task fails, the scheduler retries it according to the configured max_retries and retry_delay_seconds. If all retry attempts are exhausted, the task is marked as Failed and the failure is logged with error details. The task remains enabled and will attempt to run again at its next scheduled time.

Q: Can I run a task on multiple devices at once?

A: Yes. Backup and HealthCheck tasks can target device groups rather than individual devices. Set the target_type to device_group and provide the group ID. The task executes against every device in the group. Deployment tasks deploy a stack instance which can target multiple devices as defined in the stack configuration.

Q: How do I pause a scheduled task without deleting it?

A: Toggle the Enabled switch to off. The task remains in your task list with its full configuration and history, but the scheduler will skip it until you re-enable it. You can also use the API to set enabled: false in an update request.

Q: What is an AgentTask and how does it differ from CustomScript?

A: An AgentTask sends a natural-language prompt to the NOC agent system, which uses AI to interpret the request, gather data from devices, and produce a structured response. A CustomScript executes a script you write yourself (Python, Bash, etc.) with full control over the logic. Use AgentTask when you want AI-driven analysis; use CustomScript when you need deterministic, scripted automation.

Q: How long does task execution history persist?

A: Execution history is stored in PostgreSQL alongside the task record. Every run updates last_run_at and records detailed logs. The history is retained indefinitely unless you configure a data retention policy on your Controller instance.

Troubleshooting

Task not executing on schedule

Verify the timezone is set correctly. A task scheduled for 0 2 * * * in America/New_York runs at 2:00 AM Eastern, which is 7:00 AM UTC (or 6:00 AM UTC during daylight saving time). Check next_run_at in the task details to confirm the calculated run time matches your expectation. Also verify the cron expression is valid using the Cron Expressions syntax reference.

Task failing on specific devices

Check that the credential associated with the task is still valid and has access to the target devices. Verify SSH connectivity from the Controller host to the device. Review the task execution logs for specific error messages — common causes include expired credentials, changed device passwords, or firewall rules blocking the Controller.

Task stuck in pending state

A task stuck in pending typically indicates the job queue is backed up or the Controller scheduler process is not running. Check the Controller logs for scheduler errors. If the Controller was recently restarted, pending tasks should be picked up automatically within one minute.

Retry attempts exhausted

If a task consistently exhausts its retries, review the execution logs for the root cause. Common issues include network timeouts (increase timeout_seconds), transient device errors (increase max_retries and retry_delay_seconds), or persistent failures that require manual investigation (check device state, credential validity).

Related Features

Explore related automation and monitoring capabilities:

• Cron Expressions — Syntax reference and common patterns for scheduling tasks
• Method of Procedures (MOPs) — Multi-step operational procedures with approval workflows and rollback
• Task Monitoring — Real-time execution status, logs, and retry tracking
• NOC Agents — AI-powered agents that execute AgentTask prompts and automate incident triage
• Config Snapshots — View and compare device configurations captured by backup tasks