Sneak peak of new data quality offering: Let us know what you think

Danny Roden
Danny Roden Administrator
edited March 7 in General

Hi everyone,

In this post I'd like to provide you all with a sneak peak of a new real-time data quality enforcement offering that we're currently exploring, that we'd love to get your feedback on (complementing our existing contact data validation products). The idea centres around exposing Data Studio functionality (e.g. validation rules, cleansing logic etc) via API so that it can become a Data Quality Hub housing all the rules for both retrospective DQ cleansing/monitoring/transformation as well as for real-time data quality enforcement:

image.png

For example, it could be used to expose simple mapping rules in SalesForce and Microsoft Dynamics, so that CRM fields can enforce quality/validity using a consistent definition as is being used for bulk data monitoring too:

image.png

To help give you a sense of this capability, I've put together a short video showcasing an early preview of this functionality:

https://players.brightcove.net/87117017001/rJUPZ43a_default/index.html?videoId=6345635822112

We'd love to get your thoughts on this and help us shape the product direction going forwards so if you could spare a moment to complete the very short survey after watching this video it'd be really appreciated:

Survey button.png

https://forms.office.com/r/GKm921MnAF

Thanks!

Danny & @Chris Roy

Tagged:

Comments

  • Jaden
    Jaden Administrator

    @Sinead Galloway @Ant

  • JRowe
    JRowe Member

    Love the idea of this @Danny Roden , but not yet ready to comment on client demand in our market. We have a conversation coming up where I will check this.

  • Juha Ritvas
    Juha Ritvas Member

    Hi Danny, thanks for sharing. Are there any similar examples where systems providing Kafka stream, Aperture could listen the Kafka stream and execute near real-time validation

  • Bizzy
    Bizzy Member

    Hi Danny - with my Data Gov hat on I like the idea of 'define once' and make it available consistently across your architecture to do upfront validation. Policies and DQ rules change by Product and Business so adding that additional criteria would be essential. i.e.

    Business: Savings

    Product: Junior Saver

    Age Range >12 (Else Error) <100(Warning)

    Business: Consumer Lending

    Product: Motor Finance

    Age Range >18 (Else Error) <85 on Loan Maturity (Else Error)

    We have a complex architecture so rolling out the end point to each front end UI would be complex, but certainly would be nice to start somewhere.

  • Josh Boxer
    Josh Boxer Administrator

    @Juha Ritvas where/how is the data being captured/submitted before it makes it into Kafka. Realtime Workflow is an API so can be integrated into a web form or app or called programmatically each time a value or record needs checking or enriching. Message me if you want to discuss

  • Josh Boxer
    Josh Boxer Administrator
    edited November 19

    @Bizzy yes you could do validation and show a message to the user/staff member entering incorrect details for them to be able to fix. Also, rather than only preventing records being submitted you might flag Motor Finance of younger age say 80+ for manual review or a Junior Saver who is 11 yrs and 362 days for delayed processing, which might be a nicer experience than 'computer says no'! (not sure if these are realistic, but hopefully thought provoking of where Business folk could be more creative to hit targets and improve UX, with the correct internal tracking and approvals of course!)

Categories