Long datatype

M.Lambert
M.Lambert Contributor
in General

Hi there,

Does anyone came across a problem of the wrong datatype coming from the outputs produced by aperture that goes to Azure Data Lake?

According to the my colleague Data Engineer the outputs need to be the long data type.

Just checking if anyone knows anything about this to offer me some support.

TIA

Answers

  • Henry Simms
    Henry Simms Administrator

    Hi @M.Lambert , do you know what file or database type the data from Data Studio is being written in to? Is it a JDBC exporter (eg to SQL server) or a file exporter (eg CSV) to an ADLS location?

  • M.Lambert
    M.Lambert Contributor

    Hey @Henry Simms from Aperture Data Studio the data is exported into CSV and uploaded into Azure Data Lake Storage Container

  • Henry Simms
    Henry Simms Administrator

    Hi @M.Lambert , CSV is a plain text format that does not have datatypes. Numeric values that databases might treat as longs are exported from Data Studio to CSV in the usual way:

    image.png

    What is the problem your colleague is actually seeing?

    It's worth noting that if CSV files are opened in Excel, numbers like these are usually displayed in scientific notation, and if the file is saved the format persists:

    image.png

  • M.Lambert
    M.Lambert Contributor

    The problem is that my colleague can't querie the data, only if he changes the data type to long, so it's creating extra work.

    The strange thing is that we have 2 very similar outputs, one is coming as real and the other as long. They have been created in the exact same way.

  • Josh Boxer
    Josh Boxer Administrator

    I think from your comments, likely Data Studio is doing the same thing in each of the 2 similar outputs, but Azure Data Factory has been set up differently. Does one of the datasets in ADF have a schema set up and this one not have one? https://learn.microsoft.com/en-us/azure/data-factory/concepts-datasets-linked-services?tabs=data-factory

  • M.Lambert
    M.Lambert Contributor

    Hey guys, thanks a lot for your help. My colleague found the solution.

Categories