The Pattern: Event-Driven Workflow Orchestration
We've spent the last decade connecting SCADA systems, market APIs, field crews, and business systems across three continents. The pattern that emerged isn't what the enterprise software vendors pitch. It's a hybrid architecture where lightweight workflow engines sit at the edge, respond to operational events in real-time, and synchronize with central business systems on a completely different timescale.
This matters in energy because you're juggling two incompatible worlds: operational technology that measures time in milliseconds and business systems that batch every four hours. Most integration platforms force you to choose. This pattern lets you operate in both simultaneously.
The Problem We're Solving
Energy operations generate thousands of events per second: breaker status changes, voltage anomalies, generation forecasts, trading signals, maintenance alerts, crew assignments. Each event might trigger workflows spanning multiple systems—a single substation alarm could cascade into work orders, parts requisitions, crew dispatches, regulatory notifications, and financial settlements.
The failure modes are expensive. We've seen:
- Trading desks missing market windows because approval workflows took forty minutes to route through email
- Field crews dispatched without current asset data because the CMMS and GIS systems couldn't talk
- NERC CIP violations from manual spreadsheet-based compliance tracking
- Substation alarm floods overwhelming operators because nobody automated the first-level triage
Traditional enterprise service buses are too slow and too centralized. Cloud workflow platforms violate data sovereignty requirements. Point-to-point integrations become unmaintainable spaghetti within six months.
Solution Architecture
The pattern has three distinct layers, each with different latency requirements and failure modes.
Edge Orchestration Layer
We deploy lightweight workflow engines—typically n8n instances—at substations, generation sites, and regional operations centers. These run on industrial PCs or edge servers, completely air-gapped from the internet. They subscribe to local event streams via MQTT, OPC-UA, or Modbus and execute immediate-response workflows.
A substation n8n instance might monitor breaker status, automatically correlate multiple alarms using simple decision trees, suppress nuisance alerts, and only escalate genuine faults to human operators. Response time: 50-200ms. The entire workflow runs locally—no network latency, no cloud dependencies, no internet exposure.
Configuration is code. Every n8n workflow exports to JSON and lives in Git. We version control everything and deploy using Ansible. When a substation gets a new edge server, we pull the latest workflow definitions from our internal GitLab and deploy in minutes.
Integration Orchestration Layer
The middle layer runs in your data center or private cloud—never public cloud for OT data. This is where n8n shines brightest. We typically deploy three to five n8n instances depending on workload isolation requirements:
- OT integration instance: Connects to historians, SCADA masters, protection relays. Strict firewall rules, no outbound internet.
- IT integration instance: Handles ERPNext, OpenProject, ActiveDirectory, email systems. Can reach external APIs under controlled conditions.
- AI/analytics instance: Feeds vector databases, triggers model inference, handles high-volume log processing.
This layer transforms data between systems with incompatible schemas. A maintenance work order created in ERPNext needs to become a task in OpenProject, trigger a crew assignment, update asset records, and notify field supervisors. That's a single n8n workflow with eight nodes, executing in under 500ms.
We use webhook triggers extensively. When ERPNext creates a purchase order for critical spare parts, it fires a webhook. n8n catches it, checks inventory levels in three different warehouses via SQL queries, escalates expedited shipping if needed, and updates project timelines in OpenProject. Total latency: 300-800ms depending on database query complexity.
Business Process Layer
The top layer runs long-duration workflows measured in days or weeks: procurement approvals, regulatory compliance checklists, project planning, financial close processes. Here we use ERPNext workflow capabilities for financial processes and OpenProject for project management workflows.
n8n bridges the gap. When a capital project in OpenProject hits a milestone, n8n creates the corresponding invoice in ERPNext, updates budget tracking, and notifies stakeholders. When ERPNext processes payroll, n8n updates time tracking in OpenProject and synchronizes leave balances.
The key architectural principle: business process platforms own state and history. Workflow automation platforms handle transitions and integration. Don't try to build a work order system inside n8n. Build the work order system in ERPNext and use n8n to connect it to everything else.
Implementation Considerations
Event Schema Design
We standardize on a simple event envelope: timestamp, source system, event type, payload, correlation ID. Every system publishes events in this format. n8n workflows subscribe to event types, not specific systems.
Example: A breaker trip generates an event with type 'equipment.fault'. Three different workflows might subscribe: one for immediate operator notification, one for automatic work order creation, one for reliability metrics calculation. Each workflow processes the same event independently.
Payload schemas evolve. Use semantic versioning and always include a schema version field. Old workflows continue processing v1 events while new workflows handle v2. Migration happens gradually—no big-bang cutovers.
Idempotency and Retry Logic
Every workflow must be idempotent. We've deployed n8n workflows that process the same event five times due to message broker retries—the outcome must be identical each time. Use unique IDs to detect duplicates. Check if the work order already exists before creating it. Update operations should be true updates, not incremental modifications.
n8n's error handling is adequate but not sophisticated. We wrap critical operations in try-catch nodes and implement exponential backoff manually. Failed workflows write to a dead letter queue—another n8n workflow monitors it and alerts humans when retry counts exceed thresholds.
Security and Network Segmentation
OT network n8n instances communicate with IT network instances through a unidirectional data diode or heavily firewalled DMZ. Data flows OT to IT, never IT to OT. Control commands require out-of-band approval through physical security controls—we don't automate switching operations from business systems.
Credentials live in environment variables, never hardcoded in workflows. We use HashiCorp Vault for credential management—n8n fetches API keys and database passwords at startup. Rotate credentials quarterly and test workflow authentication failures monthly.
Observability
Every workflow logs start time, end time, event ID, and outcome. We ship these logs to a local Elasticsearch cluster (never cloud logging for OT data). Grafana dashboards show workflow execution rates, error rates, and latency distributions.
Critical workflows include instrumentation nodes that emit metrics to Prometheus. We track: events processed per minute, average processing time, retry counts, dead letter queue depth. Alert when metrics deviate from baseline.
n8n's built-in execution history helps debugging but isn't sufficient for production observability. Treat it as developer tooling, not operations monitoring.
Testing and Validation
We maintain a complete parallel environment: dev SCADA simulator, test ERPNext instance, staging n8n deployment. Every workflow change deploys to staging first. We replay production event streams through staging and validate outputs match expected results.
Load testing matters. A single substation alarm might trigger fifty simultaneous workflow executions across the system. We've crashed n8n instances by underestimating event fan-out. Test at 10x expected peak load.
Real-World Trade-offs
n8n Strengths
We chose n8n for integration orchestration after evaluating Apache Airflow, Prefect, and commercial iPaaS platforms. n8n wins on:
- Self-hosting simplicity: Single Docker container, SQLite or PostgreSQL backend, runs on modest hardware
- Visual workflow builder that non-programmers can understand: Our operations team modifies workflows directly
- 400+ pre-built integrations including obscure industrial protocols
- Webhook triggers with sub-second latency
- Reasonable pricing for enterprise support if you need it
The AI capabilities (integrated LLM nodes, vector store connectors) are genuinely useful. We use them for alarm classification, work order routing, and document analysis. Not marketing vapor.
n8n Limitations
It's not designed for high-throughput stream processing. Above 1000 events per second, you need proper message queues and stream processors (we use Kafka and Flink for that). n8n excels at medium-volume integration workflows, not big data pipelines.
Error handling is basic. Building robust retry logic requires manual effort. The execution model is single-threaded per workflow—long-running operations block other executions. We work around this by breaking workflows into smaller pieces connected via webhooks.
The learning curve is steeper than vendors admit. Visual programming sounds simple but building maintainable workflows at scale requires discipline. Without code review processes and testing standards, you'll accumulate technical debt quickly.
ERPNext Integration Patterns
ERPNext becomes your system of record for financial transactions, asset management, procurement, and HR. Its workflow engine handles approval routing for purchase orders and expense reports. Its API is comprehensive but occasionally inconsistent—budget extra time for integration testing.
We don't replicate ERPNext data into other systems. Instead, n8n queries ERPNext via API calls when workflows need current data. This avoids synchronization headaches but means ERPNext uptime directly impacts workflow reliability. Deploy ERPNext with proper high availability.
ERPNext's strength is eliminating per-user licensing costs. We've deployed it to organizations with 500+ users—the savings versus SAP or Oracle are seven figures annually. Use that budget for professional services to customize it properly.
OpenProject for Work Management
OpenProject handles project planning, work packages, and time tracking. It's less mature than ERPNext but rapidly improving. The Gantt chart functionality is legitimately good—better than we've seen in some commercial tools.
Integration is straightforward: RESTful API, webhook support, reasonable documentation. We sync work packages bidirectionally with ERPNext work orders using n8n. Field crews update status in a mobile app; that flows through n8n into both OpenProject (for project tracking) and ERPNext (for cost accounting).
The self-hosted deployment is solid but resource-intensive. Budget 8GB RAM minimum for the application server. PostgreSQL database grows quickly with attachment storage—plan for database maintenance.
The Verdict
Event-driven workflow orchestration works when you respect the architectural boundaries. Deploy lightweight automation at the edge for real-time responses. Use n8n in the integration layer to connect incompatible systems. Let proper business applications (ERPNext, OpenProject) own long-running processes and persistent state.
The pattern scales from single-site deployments to multi-national operations. We're running variations of this architecture at utilities managing 50,000+ assets and processing millions of events daily. It's not sexy—there's no blockchain, no microservices mesh, no serverless functions. It's just boring, reliable infrastructure that keeps the lights on.
Start small. Pick one painful integration—maybe work orders between your CMMS and ERP. Build that workflow in n8n. Get it running in production. Learn the failure modes. Then expand incrementally. Don't try to automate everything on day one. This is infrastructure you'll maintain for a decade—build it with that timeline in mind.
The open-source stack (n8n, ERPNext, OpenProject) costs nothing for software licenses but requires investment in skilled people. Budget for training, professional services during initial deployment, and ongoing maintenance. The total cost of ownership is still 60-70% lower than equivalent commercial platforms, but it's not free. Anyone promising zero-cost enterprise software is lying or hasn't run it in production.