DataPrep API

DataPrep performs a series of data cleansing and manipulation actions on inputs and returns the results of those actions. DataPrep actions can generate new fields, enriching the input records with extra data points. They can also overwrite existing fields, replacing messy inputs with cleansed data. Control over whether a new field is created or an existing field is overwritten lies completely with the end user.

Actions can take multiple inputs and have multiple outputs depending on the particular operation they perform. Actions can be chained together in a sequence such that the output(s) of one action becomes the input(s) of a subsequent action. The API takes the name of an action, input fields, output fields, and sometimes options that change the behavior of the action.

There are two ways to call the API, a Long-form API and a Short-form API form. Both forms provide the same functionality, except that the short-form only allows a single input record at a time while the long-form allows for batching multiple records into a single request.

List of DataPrep Actions

The list of available actions to perform are listed below. The inputs, outputs, and options for each action are included with their definition.

Clean Email

Clean Email API Action String: email Checks an email field for validity. Outputs the email on success and blank on fail. Provides light correction (e.g. gmail.co becomes gmail.com).

Examples:

{
  "inputs": [
    {
      "FirstName": "John",
      "LastName": "Smith",
      "EmailAddr": "[email protected]"
    },
    {
      "FirstName": "Jane",
      "LastName": "Williams",
      "EmailAddr": "[email protected]"
    }
  ],
  "actions": [
    {
      "name": "email",
      "inputFields": [
        "EmailAddr"
      ],
      "outputFields": [
        "EmailAddrClean"
      ],
      "options": {
        "aggressive": 1
      }
    }
  ],
  "output": [
    "FirstName",
    "LastName",
    "EmailAddrClean",
    "EmailAddr"
  ]
}

https://api.versium.com/v2/dataprep?actions[]=email:EmailAddr:1&FirstName=John&LastName=Smith&[email protected]&output=FirstName,LastName,EmailAddrClean,EmailAddr

https://api.versium.com/v2/dataprep?actions[]=email:EmailAddr:1&FirstName=Jane&LastName=Williams&[email protected]&output=FirstName,LastName,EmailAddrClean,EmailAddr

{
  "versium": {
    "version": "2.0",
    "match_counts": [],
    "num_matches": 0,
    "num_results": 1,
    "query_id": "0fe4aa159cca6853dd",
    "query_time": 0.145,
    "results": [
      {
        "FirstName": "John",
        "LastName": "Smith",
        "EmailAddrClean": "",
        "EmailAddr": "[email protected]"
      },
      {
        "FirstName": "Jane",
        "LastName": "Williams",
        "EmailAddrClean": "[email protected]",
        "EmailAddr": "[email protected]"
      }
    ]
  }
}

Clean Name

Clean name API Action String: name Cleans bad characters from a name (only allows alphabetic characters).

Clean First Name

Clean First Name API Action String: first Cleans bad characters from a first name (only allows alphabetic characters).

Clean Last Name

Clean Last Name API Action String: last Cleans bad characters from a last name (only allows alphabetic characters).

Clean Business Name

Clean Business Name API Action String: busname Cleans bad characters from a business name (only allows alphanumeric characters).

Clean City Name

Clean City Name API Action String: city Cleans bad characters from a city name (only allows alphabetic characters).

Clean State Name

Clean State Name API Action String: state Checks for a valid US state and converts to 2 letter abbreviation if possible.

US ZIP5 Check

US ZIP5 Check API Action String: uszip5 Checks whether a field looks like a 5-digit US postal code. Does NOT check for full validity (i.e. 00005 passes this check).

US ZIP9 Check & Split

US ZIP9 Check & Split API Action String: uszip9 Checks whether a field looks like a potential 9-digit US postal code and if so, splits it into two columns containing the first 5 digits and the other 4.

Clean Country Code

Clean Country Code API Action String: country Given a country name field, outputs a country code or blank if it is invalid (allows only alphabetic characters).

Clean Phone Number

Clean Phone Number API Action String: phone Checks a phone field for validity, if invalid, outputs the number if valid, or blank otherwise.

Uppercase

Uppercase API Action String: strtoupper Converts a field to all uppercase

Capitalize Words

Capitalize Words API Action String: ucwords Capitalizes the first letter of each word in a field.

Standardize & Rank Job Role

Standardize & Rank Job Role API Action String: titlerank3 Standardizes a job role field and ranks it, lower number indicates a higher rank.

Split Full Name

Split Full Name API Action String: splitfullname2 Splits a full name into separate columns for prefix, firstname, middlename, lastname, and suffix, all uppercased.

Categorize & Split Name

Categorize & Split Name API Action String: namecatparse Parses a name field into separate columns for category (individual/business/unknown), prefix, firstname, middlename, lastname, suffix, and business name if applicable.

Examples:

{
  "inputs": [
    {
      "FullName": "John Smith"
    },
    {
      "FullName": "Jane Williams"
    }
  ],
  "actions": [
    {
      "name": "namecatparse",
      "inputFields": [
        "FullName"
      ],
      "outputFields": [
        "EntityCategory",
        "Prefix",
        "First",
        "Middle",
        "Last",
        "Suffix",
        "BusName"
      ]
    }
  ],
  "output": [
    "EntityCategory",
    "First",
    "Middle",
    "Last",
    "BusName"
  ]
}

http://api.versium.com/v2/dataprep?actions[]=namecatparse:FullName:EntityCategory,Prefix,First,Middle,Last,Suffix,BusName&FullName=John Smith&output=EntityCategory,First,Middle,Last,BusName

http://api.versium.com/v2/dataprep?actions[]=namecatparse:FullName:EntityCategory,Prefix,First,Middle,Last,Suffix,BusName&FullName=Jane Williams&output=EntityCategory,First,Middle,Last,BusName

{
  "versium": {
    "version": "2.0",
    "match_counts": [],
    "num_matches": 0,
    "num_results": 1,
    "query_id": "0fe4aa159cca6853dd",
    "query_time": 0.145,
    "results": [
      {
        "EntityCategory": "Individual",
        "First": "John",
        "Middle": "",
        "Last": "Smith",
        "BusName": ""
      },
      {
        "EntityCategory": "Individual",
        "First": "Jane",
        "Middle": "",
        "Last": "Williams",
        "BusName": ""
      }
    ]
  }
}

Fix Fuzzy Location

Fix Fuzzy Location API Action String: tlilocmap Attempts to parse a fuzzy location like "Greater Seattle Area" into a meaningful location like "Seattle, WA, US".

Extract Domain

Extract Domain API Action String: domain Extracts the domain from an email.

ESP Email Check

ESP Email Check API Action String: isespfree Checks if an email address appears to be from a free Email Service Provider or not (i.e. 1 = Free ESP, 0 = Private ESP)

Public IP Address Check

Public IP Address Check API Action String: ip Checks if an IP looks like a valid public IP address, outputs blank if the check fails.

IP Address to Integer

IP Address to Integer API Action String: ip2long Converts a dotted quad IP address into an integer value. (e.g. 206.40.146.40 becomes 3458765352)

Integer to IP Address

Integer to IP Address API Action String: long2ip Converts an integer value into a dotted quad IP address. (e.g. 3458765352 becomes 206.40.146.40)

Generate Patterned Email

Generate Patterned Email API Action String: gpe Generates a probable email for a given first name, last name, and domain.

Name-To-Email Check

Name-To-Email Check API Action String: n2echeck Checks if a name appears to match an email and outputs component match count (how many name components match the email) and a weighted score. Higher scores are better.

Clean/Extract Year

Clean/Extract Year API Action String: year Attempts to extract a valid year from a field. Will filter out non-numeric characters.

Merge Date Fields

Merge Date Fields API Action String: dobmerge Merges separate year, month, and day fields into a single field.

Date Extract

Date Extract API Action String: dob Attempts to extract a date and format it as YYYYMMDD.

DateTime Merge

DateTime Merge API Action String: tsmerge Merges separate date and time fields into a single field with the format YYYY-MM-DD HH:MM:SS

Time to Hour

Time to Hour API Action String: time2hour Extracts the hour from a timestamp field.

Format DateTime

Format DateTime API Action String: timestamp Reformats a full datetime field into YYYY-MM-DD HH:MM:SS

Date Transform

Date Transform API Action String: datetransform Transforms a date field from one format to another.

Examples:

{
  "inputs": [
    {
      "FirstName": "John",
      "LastName": "Smith",
      "DOB": "01 15 1980"
    },
    {
      "FirstName": "Jane",
      "LastName": "Williams",
      "DOB": "06 24 1990"
    }
  ],
  "actions": [
    {
      "name": "datetransform",
      "inputFields": [
        "DOB"
      ],
      "outputFields": [
        "DOB"
      ],
      "options": {
        "datetransform": 4
      }
    }
  ]
}

http://api.versium.com/v2/dataprep?actions[]=datetransform:DOB:DOB:4&FirstName=John&LastName=Smith&DOB=01 15 1980

http://api.versium.com/v2/dataprep?actions[]=datetransform:DOB:DOB:4&FirstName=Jane&LastName=Williams&DOB=06 24 1990

{
  "versium": {
    "version": "2.0",
    "match_counts": [],
    "num_matches": 0,
    "num_results": 1,
    "query_id": "0fe4aa159cca6853dd",
    "query_time": 0.145,
    "results": [
      {
        "FirstName": "John",
        "LastName": "Smith",
        "DOB": "19800115",
        "EmailAddr": "[email protected]"
      },
      {
        "FirstName": "Jane",
        "LastName": "Williams",
        "DOB": "19900624",
        "EmailAddr": "[email protected]"
      }
    ]
  }
}

Filter Values

Filter Values API Action String: mvzonk Filters out values matching a certain pattern.

Stem Domain

Stem Domain

API Action String: stemdomain Outputs the domain name for an email or url stemmed without the host.

Transliterate

Transliterate

API Action String: transliterate Converts accented characters to non-accented equivalents

Hashmap

Hashmap

API Action String: hashmap Takes an input and applies a hashing algorithm with an optional salt.

Phoneline Type

Phoneline Type

API Action String: linetype-pe Returns the line type (mobile, landline, etc.) and the name of the Carrier

LibPostal Address Standardizer

LibPostal Address Standardizer

API Action String: lpaddrstd Takes one or more inputs and tries to extract a postal address and standardize it. Returns House number, Street, Unit, City, State, Zip, Country

Combine & Standardize Address

Combine & Standardize Address

API Action String: addrstd Combines and standardizes address components into a single field.

Reverse Geocode

Reverse Geocode

API Action String: reversegeocode Accepts (US-based) latitude and longitude fields and attempts to output an address for the given location.

Simple Tag Individual

Simple Tag Individual

API Action String: stindiv Takes a first name, last name, address, and zip as inputs and creates a pseudo-unique identifier for that individual.

Simple Tag Household

Simple Tag Household

API Action String: sthhld Takes a first name, last name, address, and zip as inputs and creates a pseudo-unique identifier for that household.

Zip to Congressional District

Zip to Congressional District

API Action String: z54tocongrdist Takes a Zip5 and Zip4 and returns the congressional district for that area.

Global Region

Global Region

API Action String: globalregion Takes the name of a country and outputs the region (Africa, APAC, US/CA, LATAM, etc.)

IP To Location

IP To Location

API Action String: ip2loc Takes in an IP address and outputs IP Country, IP City, IP Zip, IP ISP Name, IP Domain Name, IP Usage Type, Proxy Type, IP Block ID, IP Block Len.

Format Phone Number

Format Phone Number

API Action String: naphonefmt Formats a phone number into a selected standard format. Only works with 10-digit (or 11-digit with country code) North American phone numbers.