Learn, collaborate and solve problems with like-minded data quality enthusiasts.
You asked for a community forum that was easier to navigate, packed with expert content and with a clear way to share ideas for improvement! Christmas has truly come early, and you can now enjoy our new look community forum with much clearer and structured categories, including categories specifically for our Experianā¦
It's really important to us that we listen to your feedback and use it to inform our product direction and to improve things that frustrate you. We're always keen to hear from customers about their experience with Data Studio. There are two ways you can do this. If you have time you can book a slot to have a conversationā¦
When you navigate into any of our categories you will see a Follow button in the sidebar. This will allow you to subscribe to getting email notifications and notifications in the community about that topic. So for example if you want to be notified of new releases, navigate to the release and downloads section and clickā¦
Aperture Data Studio Release 2.12āÆis a roll-up of all features and improvements sinceāÆReleaseāÆ2.11 (May 2023). In addition, it comes with a new Notifications inbox, Two-click Profiling, Solutions, plus UX improvements, bug fixes and security updates. Download links: Windows | Linux | Find Duplicates | SDK Notificationsā¦
Subscribe to be the first to find out!
Subscribe to this category to be the first to know!
Subscribe to this category to be the first to know!
If you find a bug, the easiest and most efficient way to raise a support ticket is via the My Experian Customer Hub. Here you will also be able to track updates to your open support tickets. To get access to the My Experian Customer Hub, reach out to your local support team.
As Aperture Data Studio users, you have the ability to access eLearning for Aperture Data Studio. Here you can learn Aperture Data Studio at your own pace and at a time that suits you. It is a great way to stay on top of the latest releases and features of Aperture Data Studio, and to ensure you are using the product toā¦
Submitted on behalf of a customer. We have created many different validation rules to monitor the quality of different datasets over time. It would be useful to denote who is the owner of a specific rule in case changes need to be made in future.
Submitted on behalf of a customer. Sometimes users might be unaware that the data they are about to load already exists in a different Space. Rather than sharing it as a View we end up with multiple versions of the same data making it difficult to know which is the correct version to use. One idea is an administratorā¦
Submitted on behalf of a customer. There are tools in the workbench that are useful to analyze Find duplicates match results, but the problem we are having is when there are records matched as a Possible match within a cluster we want to manually review them and maybe allow the reviewer to set them as a definite match orā¦
We want to improve this community even more, which is why I'd love to set up some time with you to understand how we can best serve you. Set up a conversation with me using the link below: Book some time to talk about your experience with the community
Submitted on behalf of a customer. Multiple active users collaborating in a Space have created many Workflows and Views etc. It is becoming difficult to quickly find the specific item that you are looking for sometimes needing to go in and out of a few things before finding the correct item. It would be good if theā¦
I am the Head of Customer Experience at Experian Data Quality and work closely with our talented development teams across United Kingdom and Malaysia to provide you with this community site. If you have any feedback, good or constructive, please do reach out to me by sending me a direct message (via the community inbox -ā¦
Weāre so excited to hear your ideas to make Aperture Data Studio even better. š Before you get started Search for the idea to see if it exists already ā we review ideas regularly based on votes, so upvoting an existing idea and commenting with your use case is a great way to get visibility on your idea. When youāre readyā¦
Hi, I've worked at EDQ for almost 5 years now and been responsible for the user experience in Data Studio for the majority of that time. I now work across all of EDQs products ensuring that our customer experience is as good as it can be, both in product and in our wider customer experiences like our community. I live inā¦
This maintenance release contains some UX improvements, bug fixes and security updates. Compare step improvements It is now possible to update column comparisons to be less strict and ignore small changes the might not be important, depending on the use case. Bug fixes and improvements Importing/synching objects now showsā¦
Ever since Aperture Data Studioās pre-curser, Pandora, one of our most used features and most loved features of the software has been the profiling capabilities and in recent releases this has come on even further. Data profiling is fundamentally the process of rapidly evaluating data (during a discovery phase orā¦
You can easily get your data from Data Studio into Excel or Tableau using OData. OData (odata.org) is āan open protocol that allows the consumption of data via a simple and standard RESTful APIā. If you choose to turn on OData in Data Studio (v2.2.3), you will be able to link your datasets (including snapshots) to any BIā¦
This post is to raise awareness of an additional workflow step, that is increasingly being used to provide value in many different Aperture Data Studio deployments. This step allows workflows to call a REST API in a flexible yet well-controlled manner. What do you mean additional workflow step? When you install Dataā¦
Imagine you are asked to use Aperture Data Studio to generate a new field for your sales data. You have the input fields: Discount Code, Quantity and Price. The ask is to generate a new field āOffer Priceā using the following logic: How can you do this in Aperture Data Studio? The answer is to build a custom transformationā¦
Data is available from Aperture via the REST API that is not available in Aperture. Full details of Spaces, User Groups, Users, Jobs, Datasets etc. We would like to use Aperture to manage Aperture. For example: An automated workflow which checks for long running jobs and notifies us of issues; a regular review of userā¦
Hi there, I would like to understand better about the harmonizing of addresses. If I choose the best complete option as a priority, this will mix up different addresses? I have a case where an address in London was chosen but it added Oxford as this was in a extra column for an older address. Have you had this problemā¦
This maintenance release contains some important bug fixes and security updates. Anyone running version 2.12.4/5/6/7 is strongly recommended to upgrade to this version (or a later version) to prevent issues in future. Bug fixes and improvements Re-usable Workflows now allow you to view column mappings and update if needed.ā¦
Aperture Data Studio 2.12.7 This release contains Visualize associations, plus some bug fixes and security updates. Visualize associations āAssociationsā can be selected from the list screen of any object. The options show places that the object is āUsed inā and other objects that āDepend onā the selected object. A newā¦
A question that cropped up earlier today was: 'how can we avoid different users connecting to the same data in different spaces?' Depending on how you've setup Data Studio and the processes around how your users work with it, this can be a bit of a challenge to tackle at the moment. However there's a solution, which isā¦
Hi Ian, Looks like after the recent upgrade we have an issue that experianmatch path got changed to the C drive automatically, meaning it did not following the existing configuration we have which were to be this onto D drive. And even after changing the path manually on Find Duplicates ini file, it still does not updateā¦
Hi there, Any idea why in the matching results the big clusters are coming with different cluster id? When I look in the workbench they match perfectly but the results show different cluster ids. TIA Mariana
Hi, We have the disk space issue once more for our data drive. We changed the index expiry settings to 8 days, previously it was 15 days, last time we resolved this. There is still huge temp folder, this is to ask you what can be removed from this folder. 1,4 TB is temp, can we remove dump***.dat files in temp foldersā¦
Last week we welcomed over 40 customers to our latest Data Quality User Group. Held in our London office and online the session was a chance to: hear about product updates, find out about our recent user experience research, view live demos and hear from Shawbrook Bank on how theyāve implemented Aperture Data Studio. Oneā¦
Standardise Country The function uses reference data (contained within the .dmxd file) toĀ standardise known country aliases to theĀ standard form (e.g. England = United Kingdom). See below for a preview of the logical definition of the function and a preview of the dataset that it comes with. If, after processing your data,ā¦
Is there any Official Data Studio Certification?
I am pleased to confirm that our next User Group event for Aperture Data Studio usersĀ will take place onĀ Thursday 9th November, 11am-3pm (UK-time). We will be running the event from our offices in London (near Victoria statioon) but will have a dial-in option for those who are unable to be there in person. Following onā¦
Hello dear community, We try to list all dataset in all spaces with their last refresh date. Our Aperture version id 2.8.8 We noticed that the rest API "datasets" allow us to get in it's json response: datasets : information of dataset in spaces columns: ⦠tables: information of dataset (rows account, name ā¦) batches:ā¦
I am using the 'equals' function to compare two fields, looking for differences. However, it seems like the function ignores special characters such as . @ < and this is causing a problem when I am trying to compare two fields so show email cleansing. E.g. says: email.address@place.com is equal to email.address@plac.ecomā¦
I was using the address validation function to validate StreetTown and PostTown against the UPRN on a small subset of data. For some of the records, the street town is different from Royal Mail information. For example: UPRN 1 - StreetTown: Glengolly - PostTown: Thurso UPRN 2 (next door to UPRN 1) - StreetTown: Thurso -ā¦
Hi there, Can anyone help with the harmonizing process? I have a huge list with names, phone, email, addressā¦I want the most recent record in case someone change email or address but would like the system to include information that are found in other records but not on the most recent, does that make any sense? I canā¦
This release contains an Address validation upgrade, plus some bug fixes and security updates. Validate addresses The Validate addresses step has been updated to the latest version of Batch, which means support for additional datasets, increased match rates by as much as 8% and the latest version of CorrectAddressā¦
Last week a few of us from Experian (Fran, Nicole, @Job, @Josh Boxer, @Doug & I) attended RevAlation, a three day event in London hosted by one of our technology partners, Alation A common discussion point was everyone driving their Data Governance agenda were also tackling Data Quality. Whilst Data Governance continues toā¦
I recently needed to change the default Language and Region setting used by Data Studio when adding a new Dataset: The Language and Region setting controls how date and numeric values are parsed on load (if detecting data types). For example: Dates:Ā English (United Kingdom)Ā will parse the value "01\02\2020" as 1st Februaryā¦
I'm in the process of generating a dataset from a snapshot, and I'm interested in recommendations for eliminating rows that contain null values in a particular column. Any advice would be appreciated.
If you have ideas for future challenges/puzzles/scenarios, please let us know as we're always looking for new ideas!
Hi All, We need to check and validate the Postal codes in our database for all the countries and I see that we have these built-in format validations for these countries: Do you know if there is a library in Aperture for regex codes to check the Postal code format for all the countries? Many thanks! Kind regards, Vera
Hi, is there a way to see what datasets were imported as flat files or as delimited? And for the flat files, is there anyway to adjust the metafile values or see what the metafile is? Thanks
This maintenance release contains some UX improvements, bug fixes and security updates. Bug fixes and improvements: The vertical gap between Functions has been reduced making more of the Function fit on the screen: The map on Function and Workflow builders will show the step/Function/Variable name on hover and will jump toā¦
Hello, We use Aperture 2.8.8 version, we've noticed that in the uat env, we can create an API key with more that 10000 days But in the prod we can not use more that 365 days I guess that it's related to the configuration. Could we know where can we extend the key duration please? Best regards
This maintenance release contains some bug fixes and security updates. Bug fixes and improvements Upload file help text has been simplified. Views used within Functions were not shown as Associations of objects using the Function. Solutions were picking up draft versions of objects. Exporting a file to S3 without includeā¦
Hi, Have you consider having a flag in your api to execute Workflows to use the auto Mapping columns we can have in Studio ? This will be great to avoid case sensitive headers in DataSet for instance. Regards,
I have a set of data and would like to be able to export it as an XML. I have a solution that allows me to export a set of records surrounded by XML tags and prefixed by a sort key. The different sets of records are exported into a directory and the solution then ingests these, sorts them according to the key, strips offā¦
This maintenance release contains some bug fixes and security updates. Bug fixes and improvements New icons in the left navigation menu. Validate Address step with Custom Layouts behaved differently after an upgrade. Using multiple Validate address steps in a Workflow caused duplicate columns IDs. A Workflow was able toā¦
I have a grouping of data that I have an aggregation of (sum) for various data points (about 60 rows). Could I add a final row that provides a total for each of the aggregation columns (as seen in Excel). Thanks
Why I am trying to create a solution all the spaces where I have access have been selected automatically but I want to do with 2 spaces only. Then I am trying to deselect the other spaces but its taking forever and page is then hanging after some time. Just buffering nothing really happens after that
Hi I want to fill a Dataset on scheduled basis from an oData source, but no oData options are available in my External Systems section of Aperture. I think I need to download and configure a JDBC oData driver, or am I missing something in Aperture? Many Thanks Luke
The Autonomous REST Connector JDBC driver packaged with Aperture allows you to load (and refresh) the results returned from REST API calls as Datasets. OData is a standardized REST interface, which means that any data that can be queried via OData API can be loaded into Aperture. This article will demonstrate how to useā¦
Hi DQ Community, I have a workflow that I can't run or publish because I get error messages telling me that one or more workflow steps are invalid, or that the workflow is not in an executable state. I've been through the workflow multiple times and all the steps are valid - I can open the final step and view results, andā¦
This release contains improvements to Find duplicates, the ability to switch to full screen mode, plus some bug fixes and security updates. Full screen mode When using Workflow step settings screens or Exploring data, users with a smaller screen can find it difficult. Full screen mode is a new button in the top-rightā¦
Hello dear community, my use case is: we refresh more that 200 dataset ( from SAP source) in the same refresh workflow. For reasons that we don't know some of these dataset are not being refresh sometime. How can I track the list of dataset the should be refresh but are not ? The issue is that the refresh workflow is notā¦
Good morning, I'm trying to build a logic that will compare two date values and returns the comparison as a positive or negative number, the restriction is that the check has to be done within a group that have the same ID number. However, both date values are not on the same column or row (see example below). The idea isā¦
This release contains improved Function search, the ability to copy Spaces, plus some bug fixes and security updates. Copy Spaces Like other objects, it is now possible to clone any user-defined Space and all the settings and objects within that Space. Bug fixes and improvements Function search returns matches moreā¦
This release contains 2-click profiling, plus some UX improvements, bug fixes and security updates. Default Profile output It is now possible to select one of your Profile definitions as the default for the Space (or all Spaces if created in the System Space). The default output columns will be automatically selected whenā¦
Good afternoon, I'm trying to replace specific values (sample below) from a text field in a dataset, I've highlighted the ones that I was expecting to be changed on this sample; to do that I've created a lookup column with the value and the replacement (see sample below). However, when I a run the Replace Matches functionā¦
I have a significantly large file over 5 million records. I wanted to see if there is a way to drill down in the file to see if there are any duplicate customer account numbers existing in the file
Hello, Has anyone yet used Cosmos db in Experian? We are trying to denormalize cosmos db into SQL; Is there a way in Experian we can transform the cosmos data to readable sql?
The results on my duplicate step are coming with some empty gaps and when I checked it all the gaps were the Action Affected in this column. Can someone explain to me what does this actually mean? Am I missing any data or this is just an extra field and I can remove all the Affected rows? There are 2 things in this columnā¦
Hello, I'm trying to filter or separate a few columns from the datasets so, is there a way in Experian we can separate columns?
How can I get a simple SQL running when everything is set up . the Autonomous REST connection is success. But when I try to execute SQL to add new data sets, I am getting this error, and experian documentation here, has not explicitly mentioned anything how to get those running. I checked documentation there, already, andā¦
In this article I want to share a few approaches to configuring paging when accessing REST APIs using Aperture Data Studio's Autonomous REST (AutoREST) connector. What is the AutoREST connector? If you're reading this, you're probably already familiar with the AutoREST connector, but in case you're new to it, check out:ā¦
We know how to schedule workflows to run automatically. However, once the subsequent snapshots have been produced, is there a way to notify PowerBI that the data has been updated? Currently the PowerBI developer is having to refresh the datasets manually once they know the job has completed as per the schedule.
We have scheduled refresh Jobs to refresh the Tables on daily basis. The Wfs are getting the status completed but the tables are actually not getting refreshed. I tried to manually refesh as well but that is also not working. This is happening when there is new column added in the tables and that tables is not gettingā¦
Today Iād like to share with you a reusable workflow that I built a little while ago not only as a downloadable .dmx file (see below) for you to explore/use as you wish, but also as a means of demonstrating one of the key themes of Data Studioā¦reusability. If you find that there are duplicate tasks being executed on yourā¦
Hello dear community, Our use case is as follow: We will have a big dataset and we would like to get access into it using Odata, but we are not interested in the hole dataset, only a small part. Is it possible to apply a filter when we get access to a dataset using Odata Thank you in advance
This release contains improvements to Dashboards, the Rows to columns step, plus some bug fixes and security updates. Rows to columns Additional options on Rows to columns step to aggregate values: First,Ā Sum, List etc. (rather than having to follow with a Group step).ā¦
Hi there community, I am getting this error when trying to edit some workflows. I schedule some workflows to run through the night and it failed, it was a large amount of data, but since then this error is coming up. Does anyone ever been through this and know what to do? Many thanks, Mariana
We are getting error in the execution of Experian scheduled jobs as "There is not enough space on the disk" We have installed the data from the Experian Aperture Data Studio on the E Drive (511 GB) out of which only 19 GB is remaining. Majority of the data is occupied in the E:\ApertureDataStudio\data\resource folder underā¦
Hi! Is there a way to remove the columns which do not contain any data in a workflow step? I would like to only include columns that contain data but it may change over time and would like to automate it. Many thanks! Vera
Good morning, We are currently on the beginnings of a project where we need to retrieve data from an external system using API. We have develop an small script (see below) but I would like to get some advise before running it, the idea is to collect the datasets from the external system an import them to a Aperture spaceā¦
I am attempting to manually execute a recently created workflow, but I am encountering an error. The message states that 'source 1' and 'source 2' must be provided to run the workflow, despite all my data coming from a view. I am unsure about the specific requirements it is asking for. Could someone please shed light onā¦
This release contains Data Tags as Profile output columns plus some bug fixes and security updates. Profile Data tags The All statistics Profile output now includes two new columns: Data Tags: a comma separated list of all tags assigned to the column Sensitive Data Tags: a comma separated list of all tags assigned to theā¦
Hi All! Could you please let me know how can I automatically add the input source filename to the export filename in the workflow? I can get the input filename with the metadata, but how can I add somehow as a parameter to make it the exported file's name? Many thanks! Kind regards, Vera
Note that the data below is business data and not PII. Issue: When I import the CSV file created in Aperture into PowerCurve Assisted Strategy Design, it does not recognise the values in the CSV file as being text āTRUEā or āFALSEā. If I copy the values in this column and paste them into the same column as Values, then itā¦
This maintenance release contains some UX improvements plus some bug fixes and security updates. Bug fixes and improvements Protected Excel data source was not allowing credentials to be updated Easier to map SSL certificate for Docker containers Grouping records after the Address Validation step error Output does notā¦
I am in Aperture Version 2.8.8.27 I am trying to get connected to Googlesheet thru BigQuery. BigQuery is set up with link to Googlesheet and connection to BigQuery from Aperture seems to work ok. External system Connection is OK, using Service Account with .jason Private Key File. But when I try to add dataset in apertureā¦
i am trying to call an API from an external website my agenda is to pass some IDs that will be taken as input in the workflow and the workflow will run for those IDs only and finally if i could export the result in some kind of file (csv,excel,xml)
This maintenance release contains some UX improvements plus some bug fixes and security updates. Bug fixes and improvements For North American customers, the Metro2 file parser now gives the option to generate separate Datasets for the Header and Trailer lines. Refreshing data in a multi-part file now has clearer messagingā¦
Does anyone use Data Arc with a copy of a FiServ Metro2 File? We are setting this up as net new and it appears that FiServ does not include the RDW in the header or the file. This is limiting our ability to upload the dataset into the tool. Has anyone had any experience with this?
Hello, We thinking to connect the cosmos db using a JDBC connection in our ADS. However, we did not see cosmos db as an option on the dropdown list in the external system setting. So, trying to see if anyone here has the answer to our question below: Do Experian ADS support Cosmos db? If Yes, has anyone here made thisā¦
Aperture Data Studio 2.11.7 This release contains updates to the Equals function, Views information panel, plus some bug fixes and security updates. Update to Equals function We combined the Equals and Equals(strict) functions into one for easier use. This updated Equals function will compare exactly if the input value isā¦
Hi As my workflows grow and I'm introducing more Find Duplicate steps I've realised I need to start using transform steps to remove the Match Status and Cluster ID columns after I've finished Harmonising or Filtering on them. If I don't it can be confusing when trying to select the correct Match Status and Cluster IDā¦
So I am trying to connect Experian to Azure Synapse to get DQ result. So I have tried using the JDBC SQL Server Driver (SQL Server 6.0) and in addition to this changing the customer connection string to change the Select Method from cursor to direct. However when I test connection all I get as a debug on the UI isā¦
hi, In my workflow, I am connecting to SQL server 2012 using Custom JDBC jar (Microsoft JDBC Jar 12.2) and the connection string looks like belowā¦
Earlier today I received the below email from a customer which I wanted to post on here in case the answer is of use to others: I was wondering if you could help me overcome an issue I have been facing in Aperture. When uploading a file to Aperture through the āAdd Datasetā section, in the āDefine settingsā menu there is aā¦
I have a workflow that acts as shell for several workflows that run in sequence (first cleaning data, the second evaluating that cleaning). I have set up a snapshot to capture the data after the cleaning to make sure everything is working OK. When I run the workflow, this snapshot does not retain the data for the originalā¦
Hi dear community, Is it possible to refresh a dataset data through REST web service ? I know that we can: 1- upload a file into Aperture through REST 2- create a dataset data using the file key But I can not see how to refresh an existing dataset data with a new uploaded file ,( I don't want to use delete function becauseā¦
Hello, One of my workflow datasets is SQL query and when I pull metadata like database name, table name, and hostname; the table name is coming as 'SQL Query' instead of the actual table name. Does anyone know what is causing this issue and how do I fix it?
Aperture Data Studio 2.11.6 This maintenance release contains a number of bug fixes and security updates. Bug fixes and improvements Find duplicates store mappings can be updated when re-establishing the store Functions containing columns that had been excluded were showing as invalid Profile tooltips are retained whenā¦
Hello, I am trying to run the newly created workflow manually and it is giving me an error. It says source 1 and source 2 must be supplied to run the workflow. All of my source data are coming from view. Don't know what it is asking for? Does anyone know what is the error about and how to fix it?
Is there a way to select only the recent record while doing the data merging and selecting the golden record
Hi, Is there a scramble function (much like there was in Pandora), as that feature was great for hiding PII but keeping the cardinality of values. Thanks
Hey folks, Can anyone help me with optimising performance on Validation, Filter, and Take Snapshot Steps, and hopefully also provide some benchmarks for expected performance? We have several datasets that we are cleansing, and then separating into passing and failing records based on them containing enough valid dataā¦
Aperture Data Studio 2.11.5 This release contains Automations event filter options plus some bug fixes and security updates. Automations event filter options We improved the filter options for events so that filter properties can be used more than once and added new operators like āgreater/less thanā that can be used forā¦
Hi, I recently updated the credentials for an external system, and when refreshing some of the tables a few of the columns have been removed. Previously this hasn't been a problem because I can just configure the tables, but since we upgraded I've been getting the following message: Screenshot 1: Notification that thereā¦
We have a set up here at Speedy where we have data quality split across multiple spaces. We pull all of the DQ validations out into a central space via snapshots and views to create a kind of DQ datamart which enables us to report DQ at a company level. One of the key issues we are challenged with is that users canā¦
Join us for our upcoming festiveĀ Data Validation User Group! This particular user group caters exclusively to Experian Data Validation Solutions users. We are posting this within the Data Quality Community, as we acknowledge that certain clients also use data validation. As such, this user group may be of interest to thoseā¦