Native Function: Is Web URL

James T Sidebotham

Hi,

I am using the above function and the result for checking this site "www.californianotary.academy" comes back as bad, even though it can be accessed in a browser. Does that function expect something like ".com" rather than ".academy"?

Henry Simms

Is Web URL format is a regex-based function, and you can view and edit the regex used by clicking the crossed-arrows icon on the function:

What this shows is that the default function uses the regex:

^http[s]?://[a-zA-Z0-9-.]+.[a-zA-Z]{2,3}(/\S*)*$

https://regex101.com/ is a great tool for explaining this regex (make sure to select the "Java 8" flavor):

The regex looks like it might be unsuitable for a couple of reasons:

1. Expects leading "http://" or "https://" at the start of the string
2. TLDs (the last part of the URL) are expected to be 2-3 characters, but this hasn't been the case for a while and there are now many longer TLDs - https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_is_a_domain_name

I would suggest modifying the regex here or creating a separate function that does what you need.

Josh Boxer

Thanks Henry, we will see if the Function can be improved for more modern URLs

James T Sidebotham

thanks for that feedback!

James T Sidebotham

Would be good if it could plug into some DNS service to see the validity of the website, not just the syntax - similar to how the email checker can work.

Josh Boxer

You could probably do this with a custom step to call each site, maybe with a HEAD request to see if it is available: https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/extend-data-studio-functionality/use-the-sdk/