Unlocking the Power of Agentic Workflows at Nexla with AG2 | AG2 Blog

Unlocking the Power of Agentic Workflows at Nexla with AG2

Noel Nebu Panicker AI Engineer at Nexla
· February 11, 2025 · 4 min read

AG2 has been instrumental in helping Nexla build NOVA, our data engineer co-pilot. Nova empowers our customers to create data pipelines effortlessly using natural language in an engaging, interactive experience. Thanks to AG2's human-in-the-loop capabilities, Nexla ensures strong governance, high accuracy, and a delightful user experience—making data engineering more intuitive and efficient than ever.

— Saket Saurabh, Co-founder & CEO, Nexla

Nexla NOVA Architecture

Overview

In today's fast-paced GenAI landscape, organizations are constantly searching for smarter, more efficient ways to manage and transform data. Nexla is a platform dedicated to the automation of data engineering, enabling users to get ready-to-use data with minimal hassle. Central to Nexla's approach are Nexsets—data products that streamline the process of integrating, transforming, delivering, and monitoring data.

With the introduction of Project NOVA, we're leveraging AG2, a leading open-source agentic framework, to create powerful, production-grade agentic workflows that empower users to accomplish complex tasks with the simplicity of natural language.

The Challenge

One of the primary challenges our customers face is the time and effort required to develop and manage complex data transformations. Even with a clear vision of the final data model, data transformation is a multi-step process that can be both time-consuming and technically demanding.

The Solution: Harnessing AG2 for Project NOVA

AG2 provided us with the perfect foundation to build intelligent agents capable of handling complex data tasks far beyond basic conversational functions. This led to the creation of NOVA—Nexla Orchestrated Versatile Agents, a system designed to translate natural language into precise data transformations.

Natural Language to Transforms

NOVA's Natural Language to Transforms feature allows users to take a Nexset—a data product within Nexla—and describe, in plain language, the transformation they need. NOVA then automatically generates the required transforms, whether in Python or SQL, depending on the task.

For example, a user could simply instruct, "Compute average speed and average duration for every origin-destination pair, hourly and by day of the week." NOVA breaks down this request into a series of steps, applies the necessary transformations, and delivers the desired output.

Natural Language to ELT

Natural Language to ELT allows users to build and execute ELT pipelines simply by providing natural language instructions. Users can input one or more Nexsets, a final data model, and an optional set of instructions, and NOVA does the rest.

NOVA doesn't just generate a static script—it allows users to interactively tweak the SQL logic as they go, ensuring that the final output is exactly what they need.

Use Cases

These features are designed with a broad range of users in mind:

Data Engineers: Automate routine data transformation tasks, freeing up time to focus on more strategic initiatives.
Business Analysts: Generate insights quickly without the need for complex coding, enabling faster decision-making.
Business Users: Interact with data naturally, transforming ideas into actionable queries without requiring deep technical expertise.

Technical Architecture

At the heart of NOVA's success is a sophisticated agent architecture, powered by AG2:

Planner Agent: Analyzes user queries to determine the necessary steps for the ELT or transformation task.
Query Interpreter Agent: Translates the planner's high-level steps into actionable SQL or Python.
Data Transformer Agent: Generates the required SQL or Python logic, ensuring it aligns with the specific schema and data samples.
Evaluator Agent: Reviews the generated logic for accuracy before execution.
API Agent: Manages interactions with databases and cloud services, executing the approved logic.

Using Server-Sent Events (SSE)

An essential component of NOVA's architecture is the use of Server-Sent Events (SSE) to maintain real-time communication between the backend agents and the user interface. As the agents work through the various stages of query analysis, transformation, and execution, SSE allows NOVA to stream live updates back to the user.

Conclusion

Our progress in developing NOVA has been significantly enhanced by utilizing the AG2 open-source library. AG2's powerful capabilities have been instrumental in helping us create intelligent agents that transform how users interact with data.

Project NOVA and its features—Natural Language to Transforms and Natural Language to ELT—are just the beginning of what we believe is possible with AG2.