April 9, 2019
How “Public” Are Public Records?
illumis brings together thousands of public data sources in one place, saving users significant resources by replacing time-consuming manual processes with just one simple search. Rather than spending hours (or even days) running separate searches across hundreds of databases and websites, researchers and analysts using illumis can get the results they need in just seconds and instead focus their time and energy on analyzing the results.
Sometimes the manual research process illumis replaces is even more complicated than a lengthy list of online searches. In building out our data breach notification coverage, for instance, we saw how what is technically “public” record can be extremely difficult to actually access, and nearly impossible to do so in a way that fits the fast-paced nature of today’s world.
(For background: notifications of data breaches are a requirement in many states, compelling companies to disclose any incident in which consumers’ personal information was potentially exposed or otherwise released in an unauthorized manner. The information available on illumis is the notification that a breach occurred, not the actual data that may have been compromised.)
Some states make data breach notification records available online. These records aren’t always in an easy-to-search format, but several states do make some sort of list available. With illumis, we’ve pulled these lists together in one place so you can easily run a search across all of them, or integrate that data via our API.
After these first few states though, getting data breach records gets progressively more difficult…
Connecticut, for example, makes clear on their website that they have the data breach information available, but don’t make it available online. Instead, one must submit a formal request which usually takes a few days to turn around. This is doable but a slight inconvenience and possibly a major hurdle for a project with a tight turnaround.
South Carolina has a similar approach, but only after one navigates around the broken web page for “Data Breaches” on their website. When our team first followed up asking for more info, the first reply from the state was a link… directing us back to the broken web page. It took several more exchanges to establish that the data breach info was indeed available, but again, only after a formal request was submitted in writing.
Meanwhile, Nebraska’s website was working fine but they could only provide the data breach records on a CD sent via US Post. So our team submitted a request and waited for the CD to arrive, only to find it was encoded with some fairly uncommon file types. We were able to extract the records, but it was far from a straightforward process.
Even with CDs in the mix, the award for the most challenging state (among the states that do release data at all) easily goes to New York, where a request for data breach notifications in the last year took about a month to process and was fulfilled with a more than 20,000 page pdf document of scanned pages in a seemingly random order. Searching through this trove of pdfs for a specific company or data point is nearly impossible to do in short order and even though all the information is “public record”, the public’s actual ability to review this format of the information… questionable.
It’s valuable information, however. The data we received from South Carolina, for example, has notifications of data breaches from nearly 400 companies, with a combined impact on more than 11 million consumers. Expect more states to become available on illumis soon.
If you are interested in learning more about data breach records or would like to learn about the other types of public records available on illumis including lobbying records, campaign finance records, courts, business registrations and more, you can shoot us a note at email@example.com or request a demo of the platform here.
Please Note: This post was updated in June 2020 to reflect our company’s new name: illumis