SecureVisio Architecture: A Scalable Data Lake, NG-SIEM, and High Availability by Design

Most security teams eventually run into the same wall. The more data they feed their SIEM, the slower it runs and the faster the bill climbs — until they’re quietly forced into a bad trade: keep paying more, or start deleting the very logs they may need when an incident hits. SecureVisio is built specifically to remove that trade-off.

As an integrated SIEM / UEBA / SOAR platform, SecureVisio separates a low-cost Data Lake for storage from a high-value NG-SIEM layer for analytics. The result is an architecture where you keep all your data, pay for advanced analysis only where it actually protects you, scale horizontally as volume grows, and stay online through component failures. This article walks through how that works — the cost problem it solves, the layered design, the search performance, and the high-availability mechanisms underneath.

The core SIEM problem: storage and analysis aren’t separated

In the traditional model, storing and analyzing data are one inseparable system, priced as a whole. The customer pays the highest rate for every piece of information — even the data they never analyze. That single design decision creates three compounding problems:

• Performance degrades as data grows. The more data under analysis, the slower the system responds — and it slows down precisely when you most need a fast answer, mid-incident.
• Cost grows exponentially, not linearly. Every new system, every new data source, and every extension of the required retention period inflates the bill. Doubling your data can mean a several-fold cost increase, not a 2× one.
• One price for everything. Data needed for active analysis and data kept purely “on the shelf” cost exactly the same.

The hidden iceberg: SIEM total cost of ownership

The license fee on the invoice is only the tip of the iceberg. In a monolithic model, every cost line rises at once: license, infrastructure, scaling alongside growing data, the vendor’s architecture, and ongoing maintenance and tuning.

Vendor architecture — indexing, Hot/Warm/Cold data tiers, and whether log management is separated from the SIEM — is a decisive factor in total cost of ownership. To rein in spend, customers start cutting retention or deleting data. And that brings us back to the same bad compromise: pay more and more, or give up visibility.

The SecureVisio answer: a Data Lake separated from NG-SIEM

SecureVisio avoids the trap by splitting the architecture into two independently scalable layers:

• A Data Lake as the data layer — where everything is stored, cheaply.
• An NG-SIEM as the protection layer — where selected data is analyzed for prevention, detection of real threats, and response.

The effect is straightforward: the customer retains all data in the Data Lake and pays for advanced NG-SIEM analysis only where it genuinely protects them — never for processing data that doesn’t need it.

Why separating the layers matters

Two outcomes make this separation more than an implementation detail.

Full visibility. This proves its worth the moment an incident occurs. Breaches are often detected only months after compromise. If the logs from the period of the intrusion are missing, it becomes impossible to reconstruct the sequence of events, scope the breach, or confirm whether the threat has been fully eliminated. Cheap, complete storage keeps that history intact.

Cost control. Data volume grows with the organization — new systems, cloud migration, more sources. In monolithic architectures, that growth drives simultaneous cost increases and falling operational efficiency. Separating the storage and analytics layers lets you scale each independently and predictably: storage capacity grows linearly, while compute is added only when analytical needs actually require it.

Inside the architecture

At its core, SecureVisio uses a three-tier design — a presentation layer (GUI), an application layer (Application Server), and a data layer (Database Server) — accessed via SV Center and a browser or dedicated application. Around that core, an extensible set of services scales out as needed: Zone Collectors, XDR Agents, Flow Collectors, vulnerability scanners, a PAM service, and a playbook service, each able to grow from one to many instances. This is what makes the architecture scale in step with the environment rather than against it.

The Data Lake layer: LogForwarder and LogStore

The Data Lake is the log-management layer, built from two components — and crucially, no correlation happens here.That eliminates the problem of a SIEM wasting compute on irrelevant logs; routing is governed entirely by rules configured during deployment.

• LogForwarder defines retention policies and handles the receipt, parsing, and distribution of logs to the right repositories. Based on defined rules, it decides which repositories receive raw and normalized logs, whether data is correlated or only archived, and it can send the same data to multiple repositories in parallel.
• LogStore is the service responsible for writing logs and building and storing indexes. Engineered for efficient storage and fast search across very large volumes, it keeps both raw and normalized logs — enabling advanced analytics while meeting evidentiary, audit, and regulatory requirements.

The log-management layer runs many collectors of this type. Each handles around 18,000 EPS, enabling horizontal scaling up to 11 TB per day with the appropriate compute behind it.

Lowering TCO with the Data Lake

Adopting a Data Lake architecture significantly reduces the total cost of owning the data, through two mechanisms:

• Separating the storage and compute layers. This is the key element: the cost of storing data and the cost of analyzing it become independent, eliminating the situation where analysis automatically generates disproportionate infrastructure costs.
• Data compression. This adds further optimization, cuts disk-space requirements, and translates directly into lower cost. As a result, cost grows linearly with data volume and stays predictable, instead of escalating non-linearly with the load on the analytics system.

Rather than holding all data in an expensive SIEM tier, SecureVisio uses a cheaper, scalable data store where most of the volume operates in a “storage” model rather than an “analytical processing” one.

Fast search on a low-cost lake

Low storage cost doesn’t have to mean slow queries. The Data Lake delivers high query performance through several mechanisms working together:

• Columnar format — queries read only the required columns rather than entire records, sharply reducing the data processed.
• Time partitioning — data is divided by time, so queries touch only the relevant range instead of the entire historical set.
• Parallel processing — queries run simultaneously across many nodes, cutting analysis time for large volumes.
• Hot data tier — the freshest data lands on the fastest media, giving instant access to the most frequently used information.

In practice, searching a full week of logs for a specific string takes around 10 seconds in the worst case — that is, when there’s no match and the entire range must be analyzed. When the pattern does exist, results appear almost immediately after the first hit.

Hot / Warm / Cold tiering

Tiering data lets you match infrastructure cost to how often the data is actually used:

• Hot — the newest data, on the fastest media (SSD/RAM). It serves most current queries and returns instant responses.
• Warm — historical data with moderate access frequency, on cheaper but still capable media.
• Cold — archival data, rarely analyzed, on very inexpensive media, yet still available for audits and forensic investigations.

The NG-SIEM layer: analysis where it counts

Because rules implemented in the SIEM engine ensure only a selected portion of traffic ever reaches it, NG-SIEM eliminates unjustified consumption of compute resources. The layer is built from four key components:

• EventStore — the service that writes events; a repository of security events generated by the correlation engine and other analytical modules. Separating EventStore from the log store makes reporting, search, incident analysis, and event-lifecycle management more efficient.
• AnomalyDetector — runs anomaly analysis using machine learning (including isolation forest, statistics, and rare-value detection) and behavioral analytics with user and asset profiling (UEBA), surfacing unusual behavior and attacks that classic correlation rules miss.
• PolicyEngine — the SIEM and UEBA correlation engine, which also auto-populates the CMDB from incoming data. Correlation rules here draw on asset, vulnerability, and risk context to detect incidents precisely.
• Collector Controller — monitors the health of SecureVisio, collectors, and XDR agents, handles updates, and underpins high availability by detecting failures, switching modules to standby instances, and exposing health-check ports for monitoring systems and external load balancers.

Horizontal scaling with Zone Collectors

Scaling SecureVisio means adding Zone Collectors — autonomous nodes responsible for receiving, processing, and storing telemetry, each able to run as a separate physical, virtual, or cloud unit. Each typically handles about 18,000 EPS and operates fully independently, so throughput growth is essentially linear. Run five and you reach the order of 90,000 EPS, regardless of whether they sit on physical servers, VMs, or in the cloud.

Search scales the same way. Log queries execute in parallel across all collectors, and results are automatically aggregated into a single, coherent view in the operator console — independent of where the collectors actually sit. The payoff is striking: search speed at 90,000 EPS is the same as at 10,000 EPS.

Layer-separation security and rule-based forwarding

The full separation of the log-storage layer from the analytics layer — correlation, UEBA, and anomaly detection — can be implemented at the service level or physically, across different locations and security zones. Parsing and storing logs might happen in one location while correlation and anomaly analysis run in another.

What moves to the correlating collectors is governed by LogForwarder rules. A script-tab rule can take a form such as isTrue(frame.IsTraffic), and only qualifying rules determine which logs enter correlation analysis. Network traffic meeting a condition like IsTraffic can be saved locally in the log repository and, at the same time, forwarded remotely to another Zone Collector for correlation — while the log-management layer stays fully isolated from correlation.

Geographic separation and regulatory compliance

For organizations under frameworks such as NIS2 and DORA, this separation becomes a compliance enabler. Logs can be stored locally within a given country or region per local regulations, while selected event categories — security events, for example — are replicated to a central location for correlation and incident handling.

• Edge locations run Zone Collectors responsible for receiving, parsing, and storing logs.
• The central location runs Zone Collectors with extended correlation and analytics components (PolicyEngine, AnomalyDetector).
• LogForwarder governs which logs stay strictly local and which are replicated for central correlation.

The result is a distributed deployment that satisfies data-residency requirements without sacrificing centralized visibility.

Threat hunting with logical expressions

Threat hunting in SecureVisio is built on composing queries from logical expressions that can be freely combined and nested — no specialist query language required. Analysts work with familiar building blocks:

• Logical operators — AND to require both conditions (narrowing), OR for at least one (broadening), and NOT / != to exclude noise such as trusted processes and IPs.
• Patterns and characters — wildcards (*, %, e.g. malware), regular expressions and the . operator for variants such as obfuscated commands and C2 domains, and comparisons (==, !=, >, <, e.g. FileSize > 16).

Logical-expression queries give the analyst full control over what they’re looking for in the event repository — without rigid, pre-built filters.

AI-powered log search

Gen-AI integrated into the platform takes threat hunting further. From the log viewer, an operator can work with an AI Assistant and define detailed prompts — roles, schemas, filters, commands, HTTP requests, SQL queries, or log summaries — that drive system actions and return ready-made queries, summaries, or recommendations.

A baseline “Analyze everything” prompt produces a detailed explanation of an event, and the operator can easily shorten the response or ask for a concrete recommendation of next steps. SecureVisio can go further and suggest specific actions to reduce a given risk, laid out with the pros and cons of each approach — turning raw log lines into guided decisions.

Log storage and data integrity

Underpinning everything is a storage model built for trust. Logs are kept in both raw and normalized form in LogStore, a dedicated file-based database created for SecureVisio. Files average around 300 MB, compressed and cryptographically protected with a hash function and digital certificate. This guarantees integrity and non-repudiation — critical during audits and evidentiary proceedings — and simplifies retention management and storage-capacity planning.

Flexible data collection and parsing

SecureVisio supports both active and passive collection, covering heterogeneous IT and OT environments. Active methods include API integrations, Microsoft Security Event Log via MSRPC, database reads via ODBC, flat-file reads, email, and SecureVisio XDR agents. Passive methods include cleartext and encrypted Syslog (e.g. TLS/SSL), Windows Event Forwarding, NetFlow/sFlow/IPFIX, and webhooks. The platform also integrates with external vulnerability scanners and inventory tools (SNMP, LLDP, CDP, network scanners, WMI), automatically populating the CMDB and vulnerability-management modules.

For parsing, it handles a broad range of formats — CEF, Syslog (RFC 3164 and RFC 5424), email structure, regular expressions, LEEF, URL-based HTTP queries, XML and JSON by structure, and key-value with a definable separator — so new sources onboard quickly while data quality stays high for correlation and anomaly analysis.

High availability: a critical requirement

In a security context, high availability is treated as critical — and for concrete reasons:

• A gap in monitoring continuity creates a window an attacker can exploit.
• A failure must never mean losing the accumulated history of events.
• Logs are evidentiary material and the basis of regulatory compliance.
• The system must keep working despite component failures, through redundancy and automatic failover.

SecureVisio fully supports HA scenarios for both collectors and XDR agents, in active-active or active-passive mode depending on availability requirements and network architecture. The Collector Controller controls HA: it monitors the state of collectors and XDR agents, exposes health-check ports for monitoring systems and external load balancers, and detects failures to switch traffic to standby collectors or agents — minimizing gaps in monitoring.

HA can be driven in two ways. Autonomously, with failover logic running inside SecureVisio itself, which simplifies architecture and integration. Or through external mechanisms, where SecureVisio exposes health-check ports and an external load balancer switches traffic between primary and standby collectors on its own, including failing back once a fault clears. In both models the signal is unambiguous: closing a health-check port tells the monitoring system or load balancer that an instance is malfunctioning and should be pulled from traffic.

One platform, many functions

SecureVisio’s architecture was designed from the ground up for high availability, scalability, and deployment flexibility in complex enterprise environments. The central platform provides advanced log formatting and parsing, integrations with external systems, a dedicated file-based log store, and native HA. Autonomous Zone Collectors enable horizontal performance scaling, the separation of log management from analytics, and distributed architectures aligned with regulatory requirements.

Put together, SecureVisio unifies SIEM, UEBA, SOAR, and asset, vulnerability, and risk management in one coherent solution — one that grows with your data, stays online when components fail, and lets you keep full visibility without paying to analyze data that doesn’t need it.

Frequently asked questions

What is SecureVisio? SecureVisio is an integrated SIEM / UEBA / SOAR platform that also unifies asset, vulnerability, and risk management. Its architecture separates a low-cost Data Lake for storage from an NG-SIEM layer for analytics, so organizations keep full visibility while controlling cost.

How does separating the Data Lake from NG-SIEM reduce SIEM TCO? It decouples storage cost from analytics cost. Most data lives cheaply in the Data Lake in a “storage” model, and only selected, relevant traffic is processed by NG-SIEM. Storage capacity scales linearly while compute is added only when analytical needs require it — so cost stays predictable instead of escalating.

How does SecureVisio scale? Horizontally, by adding autonomous Zone Collectors. Each typically handles around 18,000 EPS, so five collectors reach roughly 90,000 EPS — on physical servers, VMs, or in the cloud. Importantly, search speed stays constant: querying at 90,000 EPS is as fast as at 10,000 EPS.

Why is search on the Data Lake still fast despite low storage cost? Because of columnar storage (reading only needed columns), time partitioning (scanning only the relevant range), parallel processing across nodes, and a hot-data tier on the fastest media. Searching a week of logs for a string takes around 10 seconds in the worst case.

What is the Hot/Warm/Cold model? A tiering scheme that matches infrastructure cost to access frequency: Hot for the newest data on SSD/RAM, Warm for moderately accessed historical data on cheaper media, and Cold for archival data on very inexpensive media that remains available for audits and forensics.

How does SecureVisio support high availability? HA is supported for collectors and XDR agents in active-active or active-passive mode. The Collector Controller monitors health, exposes health-check ports, detects failures, and switches traffic to standby instances — either autonomously within the platform or via an external load balancer.

How does SecureVisio help with NIS2 and DORA compliance? By separating log management from analytics, it lets organizations store logs locally per regional regulations while replicating only selected event categories to a central location for correlation — supporting data residency without losing centralized visibility. Cryptographically protected logs (hash plus digital certificate) also provide the integrity and non-repudiation needed for audits.