Informatica update targets preparing data for AI development

New capabilities

Fueled by OpenAI’s launch of ChatGPT in November 2022, which was a significant improvement in generative AI capabilities, enterprise interest in developing both traditional AI as well as generative AI tools has surged over the past two years.

Because large language models such as ChatGPT and Google Gemini enable true natural language processing, enterprises are aiming to combine their proprietary data with LLM capabilities to enable their employees to work with data using natural language rather than code. With coding skills no longer always needed to work with data, more employees within organizations can use analytics to inform decisions, making decision-making more efficient and accurate.

In addition, because LLMs can be trained with proprietary data to automate repetitive tasks that take up a significant amount of data experts’ time, AI tools can make application developers, data engineers, data scientists and other trained experts more efficient.

However, if the proprietary data used to train AI tools isn’t prepared properly, the models and applications trained on that data won’t perform as intended.

Enterprises often possess massive amounts of data, much of it unstructured such as text, images and audio files that have been loaded into a data lake or some other storage repository and left untouched. Even some structured data, such as financial and point-of-sale transaction records, is often simply loaded into a data warehouse and left alone.

To get all that data ready to inform AI tools, it needs to properly prepared, according to Gaurav Pathak, Informatica’s vice president of product management AI and metadata.

Data readiness for AI is critical since the outputs from AI are only as good as the data it is trained on. If you train AI on data that says the world is flat, that is what it will believe.

“Many organizations hold terabytes or petabytes of data, both structured and unstructured. But too much of that data has not been properly managed and governed. It’s not what we call AI-ready,” he said. “Cleaning up messy data will help enterprises prepare data for AI.”

Informatica’s “Fall Release” is intended to enable enterprise customers to clean up their messy data. One key component of the update is improved integration capabilities for data stored in Databricks and Google BigQuery, according to Catanzano.

The “Fall 2024 Release” includes an integration between Informatica’s no-code tools and Databricks’ generative AI capabilities; an SQL-based data transformation feature that enables users to process extract, load and transform (ELT) pipelines in Databricks Delta Lake and Google BigQuery; and a task wizard that guides users as they ingest and replicate data for AI projects.

“Aligning with Databricks is a solid step since they are moving quickly with their [generative AI capabilities],” Catanzano said. “As an intelligent data management platform, Informatica needs to be well integrated wherever their customers are or want to be as a management layer.”

Petrie likewise noted the significance of adding ELT pipelines to Delta Lake and BigQuery.

“The ELT enhancements make a lot of sense,” he said. “Many data teams now favor ELT … because they can perform sophisticated transformations on data after ingesting it into platforms such as Databricks and Snowflake.”

Informatica’s “Fall 2024 Release” also includes the following:

• Turbo-charged Application Integration Runtime, a feature scheduled for general availability in November that aims to improve application performance with autoscaling, high throughput and low-latency integration capabilities. It includes a serverless option.
• Prebuilt integration templates for integrating data stored in AWS, Microsoft Azure, Google and Oracle, among others.
• Connectors to various AI development environments, such as Amazon Bedrock and Google Vertex AI, as well as business and messaging applications, including Coupa, Salesforce Streaming Events and Azure Service Bus.
• New master data management features designed to improve workflow integration.
• Improved data governance capabilities, including metadata access controls in Informatica’s Cloud Data Governance and Data Catalog.
• Expanded regional availability of Claire GPT.

Petrie noted that Informatica’s update contains a wide range of new and improved features. Perhaps most significant is that they complement one another with a tool such as Turbo-charged Application Integration Runtime targeting the speed and efficiency of integrations enabled by connectors and prebuilt integration templates.

“Informatica’s enhancements for application integration make a lot of sense,” Petrie said. “To differentiate themselves, AI adopters must optimize the user experience with custom applications based on governed, well-integrated data. Informatica is helping companies do this faster and more efficiently on popular data platforms with its new templates and connectors.”

While enabling customers to prepare their data for training AI models and applications is the intent of Informatica’s latest update, the impetus for adding capabilities aimed getting data ready for AI came from a combination of customer feedback and the vendor’s own research, according to Pathak.

“Customer requirements … are always major drivers, along with our own research and development,” he said. “Today, many business and tech leaders are looking to accelerate their GenAI projects and strategic initiatives. We’re helping them with these latest innovations.”

Next steps

Although Informatica’s Fall 2024 Release focuses on enabling customers to prepare data for training AI models and applications, it does not address the actual development of AI tools other than providing integrations, Catanzano noted.

Last May, Informatica introduced a low-code/no-code environment for developing AI models and applications. Included are drag-and-drop capabilities, customizable templates, prebuilt techniques for generative AI development and support for a variety of LLMs and vector databases.

The vendor’s latest platform update includes integrations and connectors with development environments from other vendors such as Databricks and Google but does not include new and improved features for its own development environment.

As a result, Catanzano suggested that Informatica focus some of its future product development and marketing on its own tools for model and application developers.

“I think [Informatica should do] more to get customers to see that their platform is where they should be looking to build GenAI solutions,” he said. “They focus on getting your data ready, but I haven’t seen much about where to go next.”

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.