DataPrep API

DataPrep performs a series of data cleansing and manipulation actions on inputs and returns the results of those actions. DataPrep actions can generate new fields, enriching the input records with extra data points. They can also overwrite existing fields, replacing messy inputs with cleansed data. Control over whether a new field is created or an existing field is overwritten lies completely with the end user.

Actions can take multiple inputs and have multiple outputs depending on the particular operation they perform. Actions can be chained together in a sequence such that the output(s) of one action becomes the input(s) of a subsequent action. The API takes the name of an action, input fields, output fields, and sometimes options that change the behavior of the action.

There are two ways to call the API, a Long-form API and a Short-form API form. Both forms provide the same functionality, except that the short-form only allows a single input record at a time while the long-form allows for batching multiple records into a single request.

List of DataPrep Actions

The list of available actions to perform are listed below. The inputs, outputs, and options for each action are included with their definition.

Clean Email

Clean Email
API Action String: email
Checks an email field for validity. Outputs the email on success and blank on fail. Provides light correction (e.g. gmail.co becomes gmail.com).

Examples:

{
  "inputs": [
    {
      "FirstName": "John",
      "LastName": "Smith",
      "EmailAddr": "[email protected]"
    },
    {
      "FirstName": "Jane",
      "LastName": "Williams",
      "EmailAddr": "[email protected]"
    }
  ],
  "actions": [
    {
      "name": "email",
      "inputFields": [
        "EmailAddr"
      ],
      "outputFields": [
        "EmailAddrClean"
      ],
      "options": {
        "aggressive": 1
      }
    }
  ],
  "output": [
    "FirstName",
    "LastName",
    "EmailAddrClean",
    "EmailAddr"
  ]
}

https://api.versium.com/v2/dataprep?actions[]=email:EmailAddr:1&FirstName=John&LastName=Smith&[email protected]&output=FirstName,LastName,EmailAddrClean,EmailAddr

https://api.versium.com/v2/dataprep?actions[]=email:EmailAddr:1&FirstName=Jane&LastName=Williams&[email protected]&output=FirstName,LastName,EmailAddrClean,EmailAddr

{
  "versium": {
    "version": "2.0",
    "match_counts": [],
    "num_matches": 0,
    "num_results": 1,
    "query_id": "0fe4aa159cca6853dd",
    "query_time": 0.145,
    "results": [
      {
        "FirstName": "John",
        "LastName": "Smith",
        "EmailAddrClean": "",
        "EmailAddr": "[email protected]"
      },
      {
        "FirstName": "Jane",
        "LastName": "Williams",
        "EmailAddrClean": "[email protected]",
        "EmailAddr": "[email protected]"
      }
    ]
  }
}

Clean Name

Clean name
API Action String: name
Cleans bad characters from a name (only allows alphabetic characters).

Clean First Name

Clean First Name
API Action String: first
Cleans bad characters from a first name (only allows alphabetic characters).

Clean Last Name

Clean Last Name
API Action String: last
Cleans bad characters from a last name (only allows alphabetic characters).

Clean Business Name

Clean Business Name
API Action String: busname
Cleans bad characters from a business name (only allows alphanumeric characters).

Clean City Name

Clean City Name
API Action String: city
Cleans bad characters from a city name (only allows alphabetic characters).

Clean State Name

Clean State Name
API Action String: state
Checks for a valid US state and converts to 2 letter abbreviation if possible.

US ZIP5 Check

US ZIP5 Check
API Action String: uszip5
Checks whether a field looks like a 5-digit US postal code. Does NOT check for full validity (i.e. 00005 passes this check).

US ZIP9 Check & Split

US ZIP9 Check & Split
API Action String: uszip9
Checks whether a field looks like a potential 9-digit US postal code and if so, splits it into two columns containing the first 5 digits and the other 4.

Clean Country Code

Clean Country Code
API Action String: country
Given a country name field, outputs a country code or blank if it is invalid (allows only alphabetic characters).

Clean Phone Number

Clean Phone Number
API Action String: phone
Checks a phone field for validity, if invalid, outputs the number if valid, or blank otherwise.

Uppercase

Uppercase
API Action String: strtoupper
Converts a field to all uppercase

Capitalize Words

Capitalize Words
API Action String: ucwords
Capitalizes the first letter of each word in a field.

Standardize & Rank Job Role

Standardize & Rank Job Role
API Action String: titlerank3
Standardizes a job role field and ranks it, lower number indicates a higher rank.

Split Full Name

Split Full Name
API Action String: splitfullname2
Splits a full name into separate columns for prefix, firstname, middlename, lastname, and suffix, all uppercased.

Categorize & Split Name

Categorize & Split Name
API Action String: namecatparse
Parses a name field into separate columns for category (individual/business/unknown), prefix, firstname, middlename, lastname, suffix, and business name if applicable.

Examples:

{
  "inputs": [
    {
      "FullName": "John Smith"
    },
    {
      "FullName": "Jane Williams"
    }
  ],
  "actions": [
    {
      "name": "namecatparse",
      "inputFields": [
        "FullName"
      ],
      "outputFields": [
        "EntityCategory",
        "Prefix",
        "First",
        "Middle",
        "Last",
        "Suffix",
        "BusName"
      ]
    }
  ],
  "output": [
    "EntityCategory",
    "First",
    "Middle",
    "Last",
    "BusName"
  ]
}

http://api.versium.com/v2/dataprep?actions[]=namecatparse:FullName:EntityCategory,Prefix,First,Middle,Last,Suffix,BusName&FullName=John Smith&output=EntityCategory,First,Middle,Last,BusName

http://api.versium.com/v2/dataprep?actions[]=namecatparse:FullName:EntityCategory,Prefix,First,Middle,Last,Suffix,BusName&FullName=Jane Williams&output=EntityCategory,First,Middle,Last,BusName

{
  "versium": {
    "version": "2.0",
    "match_counts": [],
    "num_matches": 0,
    "num_results": 1,
    "query_id": "0fe4aa159cca6853dd",
    "query_time": 0.145,
    "results": [
      {
        "EntityCategory": "Individual",
        "First": "John",
        "Middle": "",
        "Last": "Smith",
        "BusName": ""
      },
      {
        "EntityCategory": "Individual",
        "First": "Jane",
        "Middle": "",
        "Last": "Williams",
        "BusName": ""
      }
    ]
  }
}

Fix Fuzzy Location

Fix Fuzzy Location
API Action String: tlilocmap
Attempts to parse a fuzzy location like "Greater Seattle Area" into a meaningful location like "Seattle, WA, US".

Extract Domain

Extract Domain
API Action String: domain
Extracts the domain from an email.

ESP Email Check

ESP Email Check
API Action String: isespfree
Checks if an email address appears to be from a free Email Service Provider or not (i.e. 1 = Free ESP, 0 = Private ESP)

Public IP Address Check

Public IP Address Check
API Action String: ip
Checks if an IP looks like a valid public IP address, outputs blank if the check fails.

IP Address to Integer

IP Address to Integer
API Action String: ip2long
Converts a dotted quad IP address into an integer value. (e.g. 206.40.146.40 becomes 3458765352)

Integer to IP Address

Integer to IP Address
API Action String: long2ip
Converts an integer value into a dotted quad IP address. (e.g. 3458765352 becomes 206.40.146.40)

Generate Patterned Email

Generate Patterned Email
API Action String: gpe
Generates a probable email for a given first name, last name, and domain.

Name-To-Email Check

Name-To-Email Check
API Action String: n2echeck
Checks if a name appears to match an email and outputs component match count (how many name components match the email) and a weighted score. Higher scores are better.

Clean/Extract Year

Clean/Extract Year
API Action String: year
Attempts to extract a valid year from a field. Will filter out non-numeric characters.

Merge Date Fields

Merge Date Fields
API Action String: dobmerge
Merges separate year, month, and day fields into a single field.

Date Extract

Date Extract
API Action String: dob
Attempts to extract a date and format it as YYYYMMDD.

DateTime Merge

DateTime Merge
API Action String: tsmerge
Merges separate date and time fields into a single field with the format YYYY-MM-DD HH:MM:SS

Time to Hour

Time to Hour
API Action String: time2hour
Extracts the hour from a timestamp field.

Format DateTime

Format DateTime
API Action String: timestamp
Reformats a full datetime field into YYYY-MM-DD HH:MM:SS

Date Transform

Date Transform
API Action String: datetransform
Transforms a date field from one format to another.

Examples:

{
  "inputs": [
    {
      "FirstName": "John",
      "LastName": "Smith",
      "DOB": "01 15 1980"
    },
    {
      "FirstName": "Jane",
      "LastName": "Williams",
      "DOB": "06 24 1990"
    }
  ],
  "actions": [
    {
      "name": "datetransform",
      "inputFields": [
        "DOB"
      ],
      "outputFields": [
        "DOB"
      ],
      "options": {
        "datetransform": 4
      }
    }
  ]
}

http://api.versium.com/v2/dataprep?actions[]=datetransform:DOB:DOB:4&FirstName=John&LastName=Smith&DOB=01 15 1980

http://api.versium.com/v2/dataprep?actions[]=datetransform:DOB:DOB:4&FirstName=Jane&LastName=Williams&DOB=06 24 1990

{
  "versium": {
    "version": "2.0",
    "match_counts": [],
    "num_matches": 0,
    "num_results": 1,
    "query_id": "0fe4aa159cca6853dd",
    "query_time": 0.145,
    "results": [
      {
        "FirstName": "John",
        "LastName": "Smith",
        "DOB": "19800115",
        "EmailAddr": "[email protected]"
      },
      {
        "FirstName": "Jane",
        "LastName": "Williams",
        "DOB": "19900624",
        "EmailAddr": "[email protected]"
      }
    ]
  }
}

Filter Values

Filter Values
API Action String: mvzonk
Filters out values matching a certain pattern.

Stem Domain

Stem Domain

API Action String: stemdomain
Outputs the domain name for an email or url stemmed without the host.

Transliterate

Transliterate

API Action String: transliterate
Converts accented characters to non-accented equivalents

Hashmap

Hashmap

API Action String: hashmap
Takes an input and applies a hashing algorithm with an optional salt.

Phoneline Type

Phoneline Type

API Action String: linetype-pe
Returns the line type (mobile, landline, etc.) and the name of the Carrier

LibPostal Address Standardizer

LibPostal Address Standardizer

API Action String: lpaddrstd
Takes one or more inputs and tries to extract a postal address and standardize it. Returns House number, Street, Unit, City, State, Zip, Country

Combine & Standardize Address

Combine & Standardize Address

API Action String: addrstd
Combines and standardizes address components into a single field.

Reverse Geocode

Reverse Geocode

API Action String: reversegeocode
Accepts (US-based) latitude and longitude fields and attempts to output an address for the given location.

Simple Tag Individual

Simple Tag Individual

API Action String: stindiv
Takes a first name, last name, address, and zip as inputs and creates a pseudo-unique identifier for that individual.

Simple Tag Household

Simple Tag Household

API Action String: sthhld
Takes a first name, last name, address, and zip as inputs and creates a pseudo-unique identifier for that household.

Zip to Congressional District

Zip to Congressional District

API Action String: z54tocongrdist
Takes a Zip5 and Zip4 and returns the congressional district for that area.

Global Region

Global Region

API Action String: globalregion
Takes the name of a country and outputs the region (Africa, APAC, US/CA, LATAM, etc.)

IP To Location

IP To Location

API Action String: ip2loc
Takes in an IP address and outputs IP Country, IP City, IP Zip, IP ISP Name, IP Domain Name, IP Usage Type, Proxy Type, IP Block ID, IP Block Len.

Format Phone Number

Format Phone Number

API Action String: naphonefmt
Formats a phone number into a selected standard format. Only works with 10-digit (or 11-digit with country code) North American phone numbers.