regextable: Tools for Regex Extraction and Cleaning

Description

The regextable package provides functions for extracting patterns from text using regex lookup tables and cleaning text data. It is particularly useful for text analysis tasks where you need to match multiple patterns.

Main functions

• extract - Extract pattern matches from text using regex lookup

• clean_text - Clean and normalize text for consistent matching

Datasets

• members - Lookup table of member names for regex matching

• cr2007_03_01 - Sample text data for demonstration

Author(s)

Maintainer: Shirlyn Dong shirlynd@umich.edu

Authors:

• Devin Judge-Lord

See Also

Useful links:

• https://github.com/judgelord/regextable

• https://judgelord.github.io/regextable/

• Report bugs at https://github.com/judgelord/regextable/issues

Clean Text

Description

Cleans a character vector by converting text to lowercase, removing selected punctuation (plus signs, em dashes, exclamation points), normalizing commas, and removing whitespace.

Usage

clean_text(text)

Arguments

Value

Cleaned character vector.

Examples

clean_text(c("Hello  World!", "This is\tR"))

cr2007_03_01 dataset

Description

Sample text dataset used for demonstration of regextable.

Format

A tibble with 5 columns:

date

Date of the record (YYYY-MM-DD)

speaker

Speaker name in the text

header

Header or title of the speech

url

Original URL of the source text

url_txt

Full text content from the source

Source

Generated for the regextable package.

Extract pattern matches from text

Description

Uses a regex lookup table to extract all pattern matches.

Usage

extract(
  data,
  col_name = "text",
  regex_table,
  pattern_col = "pattern",
  data_return_cols = NULL,
  regex_return_cols = NULL,
  date_col = NULL,
  date_start = NULL,
  date_end = NULL,
  remove_acronyms = FALSE,
  do_clean_text = TRUE,
  verbose = TRUE,
  cl = NULL
)

Arguments

Details

Pattern matching is performed using R's regular expression engine and is case-insensitive by default. For each input row, the function checks every pattern in regex_table and returns the first match of each pattern.

The output contains one row per pattern match per input row. If multiple patterns match the same text, multiple rows will be returned for that text.

Value

A tibble (data frame) with columns:

• row_id Integer row identifier corresponding to the input data

• Additional columns from data if data_return_cols specified

• Additional columns from regex_table if regex_return_cols specified

• pattern The matched regex pattern(s)

• match The specific text extracted from the data (original casing preserved)

Examples

# Create sample data
data <- data.frame(
  id = 1:3,
  text = c("I love apples", "Bananas are great", "Oranges and apples"),
  stringsAsFactors = FALSE
)

# Create regex patterns
patterns <- data.frame(
  pattern = c("apples", "bananas", "oranges"),
  category = c("fruit", "fruit", "fruit")
)

# Extract matches
extract(data, "text", patterns)

Extract All matches per pattern

Description

Internal function to extract matches using dual-text approach.

Usage

extract_matches_all_internal(
  text_search,
  text_raw,
  row_ids,
  patterns,
  id_col_name,
  verbose = FALSE,
  cl = NULL
)

members dataset

Description

Lookup table of member names and metadata for regex matching.

Format

A tibble with 9 columns:

congress

Congress number (numeric)

chamber

Chamber (House/President/Senate)

bioname

Full bio name of the member

pattern

Regex pattern to match this member's name

icpsr

Numeric ICPSR identifier

state_abbrev

Two-letter state abbreviation

district_code

District number (0 for President)

first_name

First name of the member

last_name

Last name of the member

Source

Generated for the regextable package.