Snowflake unveils AI development, security capabilities

New AI capabilities

Perhaps the top business benefit of generative AI is significantly increased efficiency.

Just as the telephone transformed communication a century ago and the evolution of personal computers altered how business is conducted over the past 40 years, generative AI holds the potential to revolutionize the way employees do their work.

Generative AI allows enterprises to automate myriad tasks that take up copious amounts of people’s time, enabling people to be far more productive than was previously possible.

In addition, generative AI holds the possibility of making workers smarter.

Data-driven decisions have been shown to be more effective than those made based on instinct alone. Data, however, has long been difficult to operationalize for most workers, given the complexity of data management and analytics platforms. As a result, only about 25% of employees within organizations used analytics tools as part of their decision-making process as recently as 2022.

Generative AI models, when combined with an organization’s proprietary data, eliminate much of the complexity it previously took to work with data, including the need to write code to query and analyze data and the data literacy training needed to interpret data. By enabling true natural language processing, allowing users to ask questions of their data without having to write code and delivering responses — including detailed, summarized interpretations — they help just about any employee to make data-driven decisions.

However, as much as enterprises now want to develop generative AI tools, doing so is difficult.

In response, data management and analytics vendors such as Snowflake, chief rival Databricks and myriad others have created environments for customers to build AI and machine learning tools. Snowflake’s latest new features continue that effort.

In June, the vendor unveiled Cortex Analyst and Cortex Search — neither of which was yet in preview — to help users develop conversational AI tools.

Cortex Search enables developers to extract data from unstructured text documents to feed AI models and is now generally available. Cortex Analyst, which enables developers to use structured data to inform AI tools, is now in public preview.

Beyond the two features first introduced last spring, among other new Cortex AI capabilities aimed at helping customers develop AI tools, Snowflake unveiled the following:

• Cortex Complete Multimodal Input Support so developers can inform models with unstructured data beyond just text such as images and audio files.
• Cortex Chat API to streamline integrating application interfaces with Snowflake data.
• AI Observability for LLM Apps to provide users with metrics for monitoring the quality of their data and AI applications.

Cortex Complete Multimodal Input Support is not yet in preview, while Cortex Chat API will soon be in public preview, and AI Observability for LLM Apps is now in public preview.

Perhaps the most significant new features for users are those like the LLM observability capabilities that address AI governance, according to Menninger. Just as data governance emerged as a necessity when enterprises began to adopt self-service analytics tools, AI governance is becoming increasingly needed as enterprises attempt to expand their use of AI tools.

“I am a big proponent of the trustworthy AI enhancements, including model explainability, model observability and observability for LLM applications,” Menninger said. “My recent research indicates that there is a significant lack of tooling for AI governance, leaving enterprises to fend for themselves. These new capabilities should be welcome additions to the platform.”

Additional new capabilities aimed at helping developers build AI tools include customized processing options for large batch files of text, so AI teams can create conversational AI pipelines with high processing speeds and Container Runtime to enable users to execute training workloads on graphics processing units.

Collectively, the primary benefit of the new features is speed, according to Christian Kleinerman, Snowflake’s executive vice president of product.

By providing customers with tools that make it easier for developers to discover and operationalize both structured and unstructured data as well, gain visibility into the performance of data pipeline and model health and optimize the use of compute power, Snowflake hopes to speed and simplify AI development.

“[One of] the benefits that we expect customers to realize is a much faster time to build applications that are conversationally natured for structured and unstructured data … in a trusted, cost-effective fashion,” Kleinerman said during a virtual press conference on Nov. 8.

While the new capabilities advance how customers can use Snowflake to develop AI tools, they also demonstrate that the vendor is remaining competitive with Databricks and other vendors when it comes to enabling AI development, according to Menninger.

Snowflake was slower to add such capabilities than some of its peers. But since Sridhar Ramaswamy was appointed CEO In February, the vendor has made AI development a focal point of its product development roadmap.

“All the data platform vendors are making significant investments in providing AI/ML capabilities,” Menninger said, noting that some, such as Snowflake, have taken an API-based approach that enables customers to build their own AI applications while others have provided more actual tools for their customers.

“Neither approach is right or wrong, but they appeal to different audiences,” Menninger continued. “Ideally, vendors would provide both, giving different teams the option to use the approach they prefer.”

Collaboration and security

Beyond AI development, Snowflake also introduced new collaboration and security capabilities.

New collaboration capabilities include Internal Marketplace, an environment, now generally available, where organizations can store datasets, applications and AI products so that different business units can access and use tools developed by other business units. In addition, in a move aimed at making it easier for customers to distribute analytics and AI tools, whether to other users within their organization to monetize them to sell to third parties, Snowflake’s Native APP Framework is now integrated with Snowpark Container Services.

Meanwhile, regarding security, Snowflake is adding new threat prevention and security monitoring capabilities.

Last May, using stolen user login credentials, a hacker was able to infiltrate Snowflake and steal customers’ data. Up to 165 Snowflake customers were compromised, including AT&T and Ticketmaster.

While it took a hacker to access Snowflake’s environment and the vendor’s existing security measures have prevented anyone from breaking without proper login credentials, Snowflake is nevertheless adding new security layers to try to protect customers.

Snowflake offers Horizon Catalog, a data catalog that enables users to govern their data, including its access.

To improve security, the vendor is adding Leaked Password Protection, which automatically disables any password discovered on the dark web, and Programmatic Access Tokens to authenticate APIs and prevent unauthorized access to application.

In addition to Horizon Catalog, Snowflake offers Trust Center, a tool that helps customers monitor and improve the security of their accounts. Within Trust Center, Threat Intelligence Scanner Package is now generally available to monitor which users may be risky.

Motivated by what occurred last spring, the new security features represent Snowflake’s acknowledgment that it needs to do more to protect customer accounts, according to Kleinerman.

“Even though we’ve had security capabilities for years, they were not leveraged in the appropriate way,” he said. “That has informed our view of where our responsibility ends. We now strongly believe that it’s on us to help our customers make the best use of the technologies we have. That’s why we’re adding a lot around monitoring, recommendations, insights and controls being surfaced for our customers.”

Next steps

As Snowflake plots future product development, native application development will be a focus, according to Benoit Dageville, Snowflake’s president of product and one of the vendor’s co-founders.

Native applications, such as one recently developed by Metaplane, enable customers to use other systems in connection with their Snowflake data without having to move data from Snowflake’s environment into the other system. The native connection limits exposing data to potential leaks and simplifies using two systems in concert.

“We are providing extensibility such that every use case can be performed,” Dageville said.

Menninger, meanwhile, lamented that many of the features unveiled during Snowflake Build are not yet in preview or just entering preview. While they may seem significant, they’re not yet a supported part of the vendor’s platform.

“Snowflake still needs to deliver these capabilities as generally available features,” he said.

In addition, Menninger suggested that while Snowflake’s API-first approach to AI development is attractive to some developers, adding tools beyond APIs could help the vendor appeal to a broader audience of AI developers. For example, developing generative AI-powered tools that relieve model developers of some of their work is an opportunity.

“Tooling beyond APIs would help expand its market and make AI/ML developers more efficient,” he said. “At some point, we’ll probably see GenAI dramatically simplify model development just as we see it impacting application development processes.”

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.