See Your Data in Motion: Epsio's New Dashboard Visualizes Stream Processing in Real-Time

We're excited to announce the release of our new Epsio dashboard – a feature that gives you unprecedented visibility into your Epsio deployment. Whether you're debugging latency issues, tracking view creation progress, or simply want to understand how your complex queries are processed under the hood, this dashboard provides the practical insights you need.

Technical Features You Can Use Today

The dashboard provides actionable insights at multiple levels:

Deployment Metrics Dashboard

Monitor your Epsio deployment performance with precise metrics:

• CPU and memory utilization graphs with time-series data
• Per-view latency measurements
• Change-processing throughput statistics
• Real-time health status indicators

These metrics were previously only available by running PostgreSQL commands or setting up external Datadog monitoring. Now they're accessible through a single interface, with actual numbers you can use for capacity planning and troubleshooting.

View Management Console

Take control of all your Epsio views with actionable tools:

• Status indicators showing states
• Full PostgreSQL definition access with syntax highlighting
• Performance history with min/max/avg latency values
• Direct links to view-specific visualizations

Epsio Query Visualization Engine

This is the core technical feature of the dashboard. For each view, you can now:

• See the exact operator tree Epsio creates from your query
• Track actual data volumes between processing stages (e.g., "32,500 changes processed through this JOIN")
• Identify specific operations causing bottlenecks
• View real-time processing statistics as your data changes
• Monitor view creation progress

Practical Applications

Performance Debugging

When facing latency issues, you can now:

1. Check the high-level dashboard to identify which view is problematic
2. Drill down to the specific view's visualization
3. Identify exactly which operation is processing the most data
4. Optimize your Epsio query to address the specific bottleneck

Example: A customer was experiencing increased latency. The visualization immediately showed that a JOIN was outputting changes that a later JOIN was filtering - pointing to a bad JOIN order in the query.

Resource Optimization

The dashboard enables precise resource management:

• CPU utilization graphs help right-size your Epsio instances
• Per-operation data flow statistics identify where to focus optimization efforts
• View-specific metrics help prioritize which queries need attention

View Creation Monitoring

For large datasets, view creation can take time. Now you can:

• See exactly which stage of creation is in progress
• Track progression for each operator in the Epsio view
• Understand which operations take the most time for your specific data patterns

How to Use It Today

Accessing the dashboard requires no additional setup:

1. Log in to your Epsio account
2. Navigate to your deployment page
3. Select the "Dashboard" option from the main navigation

To get the most value from the query visualization:

• Click on any operation node to see detailed statistics
• Use the timeline view to track performance changes over time
• Hover over connections between nodes to see data flow volumes
• Compare different views to identify optimization patterns

Next Steps

The new dashboard is available today for all Epsio users, running an Epsio version of 21.x.x and above using the one liner install method, or CloudFormation. We'd love to hear how you're using these new visualization tools to optimize your queries and understand your data processing better.

In future updates, we'll be adding:

• More detailed statistics per operator in the views dashboard.
• An easier way to identify bottlenecks in views other than processed diffs. 
• Custom alerting based on latency and throughput metrics.
• A Prometheus SQL Exporter for all stats.

Start exploring your query performance today!

‍