Beginner perfumer · 101
One more correction to last time's table: the assay field's coverage is not 64.5%, it is 10.5%
· 11 min read
Article 100 listed coverage for thirty fields and caught four that look 100% full and are under 12% real. Running the same sweep across every field turned up an error in my own table. I reported assay at 64.5%, but 22,771 of those 27,221 records carry the identical string '95.00 to 100.00' — 83.7% of them. Strip it and what remains is 4,450 records, 10.5%. And that default tracks obscurity almost perfectly: among records with zero suppliers 97.5% of assay values are the default, against 26.5% among those with ten or more. There is also a more complete kind of empty: the chembl field has 5,831 values, and all 5,831 of them are the word 'View' — a button on a web page, not data.
About a 4 minute read.
Article 100 listed coverage for thirty fields and caught four placeholder fields. Then I ran the same sweep across every field, and found an error in my own table.
The assay field
I reported 64.5% (27,221 records with a value).
But 22,771 of those carry the identical string "95.00 to 100.00." That is 83.7% of them.
| assay value | Records |
|---|---|
| 95.00 to 100.00 | 22,771 |
| 99.00 to 100.00 | 1,084 |
| 98.00 to 100.00 | 1,005 |
| 97.00 to 100.00 | 647 |
| 96.00 to 100.00 | 331 |
Strip that default and 4,450 records remain — 10.5% of the database.
I ranked it sixth in article 100. It belongs around eighteenth.
And the default follows a rule
It tracks almost perfectly with whether anyone sells the material:
| Records with assay | Share that are the default | |
|---|---|---|
| Zero suppliers | 13,382 | 97.5% |
| Marked natural | 12,998 | 88.0% |
| Marked synthetic | 633 | 56.6% |
| 10+ suppliers | 2,091 | 26.5% |
Materials people sell have a real purity specification three times in four. Materials nobody sells carry boilerplate thirty-nine times in forty.
(88% boilerplate among naturals makes sense too — a natural extract has no "purity"; the field was never meaningful for it.)
So the right way to read the assay field is to check the supplier count first. For well-supplied materials it is a specification; for zero-supplier records it is filler.
The second kind of empty: the field holds a button label
I missed this entirely in article 100.
| Field | With a value | Most common value | Share |
|---|---|---|---|
| chembl | 5,831 | "View" | 100.0% |
| chebi | 4,638 | "View" | 100.0% |
| fda_unii | 30,358 | "Search" | 65.0% |
| hmdb | 23,589 | "Search" | 68.8% |
In chembl and chebi, every single record says "View."
Whatever scraped these pages captured the link's anchor text instead of the identifier it pointed to. "View" is a button, not an identifier.
fda_unii and hmdb are mixed — about a third are real identifiers and two thirds are "Search."
The control: kegg, foodb, nikkaji, beilstein and mdl all have a most-common value appearing 24 times or fewer. Those are real identifiers.
The third kind: one answer for the whole field
| Field | Value | Records | Share |
|---|---|---|---|
| fcc_listed | "No" | 41,753 | 98.9% |
| "Yes" | 472 | 1.1% |
A yes/no field, 98.9% one answer. It is not broken, but it carries almost no information — unless what you want is precisely those 472.
The corrected table
| Field | Apparent coverage | After stripping boilerplate |
|---|---|---|
| notes | 100.0% | 20.9% |
| toxicity_oral | 100.0% | 12.3% |
| toxicity_dermal | 100.0% | 6.2% |
| toxicity_inhalation | 100.0% | 1.7% |
| assay | 64.5% | 10.5% |
| fda_unii | 71.9% | 25.1% |
| hmdb | 55.9% | 17.4% |
| chembl | 13.8% | 0.0% |
| chebi | 11.0% | 0.0% |
| fcc_listed | 100.0% | 1.1% |
| occurrence | 100.0% | 51.4% |
| formula | 51.9% | 51.2% |
The last row is the control: formula has 295 records reading "unspecified" and barely moves. Not every field has this problem.
Three ways to fill a field with nothing
- A placeholder —
Not determined,None found. It says outright that there is nothing. - A default template — "95.00 to 100.00". It looks like a number, and a plausible one. This is the dangerous kind.
- Mis-scraped interface text —
View,Search. Not even a number.
The second is the hardest to catch because it passes every shape check: a valid range, correctly ordered, physically plausible. None of the three checks that need no chemistry would find it.
There is one way to catch it: count how many distinct values the field holds.
In practice
- Before quoting a field's coverage, count its distinct values. If the most common one accounts for four fifths, that coverage figure is false.
- Read the assay field alongside the supplier count.
- Ignore
chemblandchebi. - This sweep is cheap — one
GROUP BY— and it caught something a hundred articles had not.
What this doesn't establish
- I only swept strings under 45 characters occurring 200 or more times. Longer boilerplate is missed.
- "95.00 to 100.00 is a default" is an inference. It is possible that twenty-odd thousand materials genuinely specify that range — but 97.5% of zero-supplier records having it makes me disbelieve that.
- I did not check the source site, so "View is mis-scraped button text" is inferred from the pattern.
- The corrected coverage figures mean "not boilerplate", not "correct". A cell containing a real number does not make that number right.
- This is a September 2026 snapshot.