The launch of MANTA release 35: automated lineage for Kafka and SAP HANA, enhancements to existing Snowflake and Power BI, and more.

Gary Allemann – MD at Master Data Management

MANTA release 3.5 includes automated lineage for Kafka and SAP HANA, enhancements to existing Snowflake and Power BI scanners, and various usability features.

Master Data Management (MDM) partner, MANTA, has released version 35 of its unified lineage platform. The MANTA platform automatically scans your organisation’s data environment to build a powerful map of all data flows and deliver it through a non-technical User Interface (UI) to give them full visibility and control of their data pipelines. Release 35 adds two popular, modern data sources, extends existing scanners for improved lineage, and simplifies administration and access management.

“Data lineage is important to any organisation wishing to deliver trusted analytics timeously, yet it is impossible to achieve using manual methods of documentation,” says Gary Allemann, MD at Master Data Management. “MANTA’s track record in adding emerging sources to their already impressive list of scanners helps to future proof any investment made by customers as they modernise their data landscapes.”

Introducing new scanners for modern technologies

Release 35 introduces scanners for two popular new data sources:

The new scanner for Kafka includes support for StreamSets and DataStage scanners. The Kafka scanner supports metadata harvesting from the Confluent registry and the ability to define a full Kafka server and topic hierarchy with a detailed metadata definition as described by Avro and JSON Schemas.

MANTA connecting to Confluent Platform Schema Registry for extracting the schemas contained in Kafka topics.

Additionally, release 35 adds lineage scanning for SAP Hana, which includes the automated collection of dictionary metadata and external script analysis via lineage parsing for independent SQL scripts in a separate sub-directory.

Enhancements to Snowflake and Power BI scanners

Release 35 brings several improvements and enhancements to existing scanners. After including support for COPY INTO attributes in Snowflake as part of Release 34, this release brings new capabilities to the Snowflake scanner.

Most significantly, they have added scanning support for transformation expressions, which are recognised by MANTA Active Tags and provide visibility for business rules across the data pipeline. This gives MANTA the ability to display business tools that are part of the Snowflake investment.

MANTA support for parsing and analysis of Snowflake Transformation Expressions

Release 35 also includes an update to the MANTA Power BI Scanner complementing Microsoft’s enhancements to the Power BI Admin API. This update provides for better access to metadata, resulting in a more complete data lineage.

Enhanced Administrative capabilities

Besides these new capabilities, new scanners, and updated scanners, Release 35 includes equally important architecture changes to improve the overall support and management of the MANTA platform.

Enabled Event Notification

The MANTA Event Framework introduces event management and notification capabilities that enable communication of critical events to be more easily highlighted. The Event Framework includes built-in connectors for industry-standard notification methodologies to, for example, notify analytics teams when new lineage metadata becomes available, or to notify support staff if a scan fails for some reason.

MANTA Event Framework

Integrated Keycloak Identity Management Solution

MANTA’s Keycloak Identity Management Solution provides for improved security capabilities and enterprise-level support for user management, including Single Sign-On (SSO) for one user-ID access to all functions, enhanced user account management, active directory, and SAML integration.

Simplified Upgrade Process

Release 35 introduces changes to how site-specific settings and configurations for MANTA Flow are managed. User-set values have been separated from product-default settings. This choice of a user-set or product-based configuration simplifies future MANTA upgrades as well as backups and migration of the configuration across environments.