Thoughts

Enhancing Metadata Management through Machine Learning

by Adam Milward , February 23rd, 2021

Machine learning is on the shopping list for many seeking IT solutions today.  The recent Gartner Magic Quadrant for Metadata Management Solutions report for 2020 acknowledged that whilst market expectations for augmentation and machine learning in metadata management solutions are accelerating, the market is largely yet to realise the potential use cases and benefits of this technology.

MetadataWorks, supported by funding from InnovateUK, is investigating the potential for machine learning to support the metadata onboarding process, and contribute to the standardisation of dataset attributes across organisations.

Specifically, MetadataWorks are investigating the ability of unsupervised natural language processing algorithms and neural networks to suggest mappings between dataset data attributes (extracted using open-source data profiling tools), and common data models, such as OMOP or the NHS Data Dictionary.

If successful, these tools could support data analysts and scientists to generate data documentation.  Additionally, the tools could demonstrate the potential to transform datasets to industry and international standards, reducing data complexity and increasing data utilisation.

An alternative use case for these machine learning techniques could support organisations to identify the range of data attributes used across organisational data, and the potential for consolidation and standardisation.

We’re looking forward to presenting the results of this work in mid 2021.  If you’re interested in seeing how these tools could work with your data, please get in touch!

Related Insights

We use Cookies

We use tools, such as cookies, to enable essential services and functionality on our site and to collect data on how visitors interact with our site, products and services. By opting in or continuing to use this site, you agree to our use of these tools for advertising and analytics.