Optimizing Search Logic and Noise Handling in Data Processing
Explore the complexities of search logic and noise words in data processing, with insights from experts in the field. Discover how policies and special characters impact information management. Learn about strategies for name normalization and unique rules for handling individual names in data systems.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Search Logic & Noise Words: How to Proceed when Making Changes Ben Ark, National Corporate Research, Ltd. Wally Boggus, Capitol Services, Inc. Darrell Pierce, Dykema Gossett PLLC/NPRRA #IACA14
Search Logic Policy Issues Circa 1994, IACA and Filing Project investigated possible search logics Cost was a huge issue It would be so much easier if we could only get filers to get the debtor name right! Strict Search Logic Prevailed But there had to be some wiggle room , so noise words and The to be disregarded But based on old paradigm Computer does not care about the Typos in noise words and the mean they are NOT disregarded #IACA14
Special Characters: , , , , , , etc. 2001 MARs: 202 As used herein, the term legible is not limited to refer only to written expressions on paper: it requires a machine-readable transmission for electronic transmissions and an otherwise readily decipherable transmission in other cases. Rule begged the question 2007 MARs: 101.7 Regardless of the method of delivery, information in UCC records may be communicated to the filing office only in the form of characters that are included in [the American Standard Code for Information Interchange character set, as more particularly described in ISO-14962- 1997 and ANSI-X3.4-1986(R1997)]. #IACA14
Special Characters: , , , , , , etc. 2007 MARs: If a UCC record is tendered that provides characters not permitted under rule 101.7 on a paper-based form, and the filing office nevertheless accepts the form for filing, the filing office will substitute, for each such character, the character [*] in its information management system. Problem with no substitution rule: Filer cannot provide debtor name as it is under 9-503, as filer is forced to use substitute characters Technically, 9-506 rule does not work either because debtor s correct name cannot be searched #IACA14
Search Logic Name Normalization 1. 2. 3. 4. Case-insensitive (A = a) Ampersand (&) turned into AND Punctuation removed Accents disregarded Accent-insensitive OR accented-chars removed? Topic of previous IACA work groups Noise endings disregarded 1. Longer to shorter (National Bank before Bank) 2. Repeat until none are removed Beginning THE removed Spaces removed 5. 6. 7. #IACA14
Search Logic Individual Names Individual names have additional special rules for handling initials for first and additional (middle) names First and additional initials equivalent to names starting with those initials Search = John Aaron Smith Surname = Smith First = John or J Additional = Aaron or A or no name Results may include John Aaron Smith John A Smith J A Smith John Smith J Smith #IACA14
Search Logic Individual Names Search = J A Smith Results may include John Aaron Smith John A Smith J A Smith John Allen Smith Jonathon Aaron Smith Josh A Smith J Smith John Smith Josh Smith #IACA14
Adding Special Characters 2013 MARS says to disregard accents Should you transliterate them into base A-Z characters? E, n, AE Should you treat them as a wildcard? This was the subject of previous STS panels Recommendation was to be consistent and publish how you are handling special characters so your filers and searchers know. #IACA14
Example of Normalization Search: The ABC & XYZ C mpany, Inc. 1. THE ABC & XYZ C MPANY, INC. Make it all uppercase 2. THE ABC AND XYZ C MPANY, INC. Replace & with AND 3. THE ABC AND XYZ C MPANY INC Remove punctuation 4. THE ABC AND XYZ COMPANY INC Disregard accents; assuming that we re transliterating 5. THE ABC AND XYZ COMPANY Remove INC noise word ending 6. THE ABC AND XYZ Remove COMPANY noise word ending 7. ABC AND XYZ Removing leading THE 8. ABCANDXYZ Remove any spaces #IACA14
A Fun Example Search: The Corporation Company, As Trustee 1. THE CORPORATION COMPANY, AS TRUSTEE Make all uppercase 2. THE CORPORATION COMPANY AS TRUSTEE Remove punctuation 3. THE CORPORATION COMPANY Remove AS TRUSTEE noise word ending 4. THE CORPORATION Remove CORPORATION noise word ending 5. THE Remove leading THE 6. Nothing If you do a partial match search, you may just return EVERYTHING! #IACA14
Search Logic Change Management Without regard to policy questions, any change to database or standard search logic that affects outcomes can create legal problems Expansion: E.g., nickname table, adding noise words, alternative spelling equivalents Causes financing statements that were not legally effective to become effective under 9-506: Effective as of when change date or initial filing date? Conundrum: statute says initial filing date but makes sense to treat as debtor add, effective from change date Don t know how courts will handle #IACA14
Search Logic Change Management Constriction: E.g., eliminating table of noise word endings Makes a financing statement that was effective under 9- 506 no longer findable in a standard search logic search Conundrum: does this make legally effective filing now ineffective, or does it instead create a hidden lien problem? Don t know how courts will handle Has there been a taking without due process? How can searchers protect themselves search all name endings? FOOSL View: Strict search logic is preferred, and stable results are more important than the exact rules applied #IACA14
Search Logic Change Management Programming Solution: Existing search logic applies to existing debtor names, and new logic applies only to names filed after change is effective Maintains stability of results for existing financing statements and avoids the taking and hidden lien problems Be wary of unintended changes in results Possible issue with multiple (e.g., in-house versus on-line) systems Potential differences based on different software language, differences in database structure, etc.? #IACA14
