We used multiple labs (as I couldn't believe what bullsh*t I was receiving), both univeristy AND commercial. I'm not mentioning names, as I'm not here to slam anyone, I'm just going by what actually happened to myself and others in the industry. The reports are somewhere around here, I probably made paper airplanes out of them, as that's about all they were good for.
-
If you need help identifying a pepper, disease, or plant issue, please post in Identification.
scovilles Let's talk Scoville scale
- Thread starter Lucky Dog Hot Sauce
- Start date
HPLC can be EXTREMELY precise. The question is: what protocol did these labs run? If they follow identical protocols, they should get statistically equivalent results. If you find a lab that follows good laboratory practises (i.e. "GLP compliant"), and you have a validated method, you will get results that are consistent, period. If you send them out to any old lab and just tell them to do the best they can, especially with natural product analysis, you're almost certainly going to get rarified crap.
That's all fine and dandy. Like I said, this is from personal experience, from different labs (even one connected to the D.O.D.), it happened to other manufacturers as well. Trust me dude, I don't pay that kind of money to send the stuff to a garage lab, and tell them to do the 'best they can', if that's what you are implying.
Your first sentence speaks volumes by the way. "HPLC can be extremely precise", "can" being the operative word, meaning not always.
Your first sentence speaks volumes by the way. "HPLC can be extremely precise", "can" being the operative word, meaning not always.
Well yes, it can be, and that word choice wasn't at all by accident. I would also add that that for maximum precision, chromatography is also your only hope. The alternative is... what? A magic eight ball? A ouija board? Divining the answer from chicken innards?
I'm not saying that you sent them to bad labs, but I am telling you that unless they run the same protocols, there is no reason on Earth to believe that they will get similar results. They need to use the same sample prep, same columns, same flow rates, same eluents, same sample loadings, same temperatures, same internal standards, etc. What's more, HPLC is only a separation technique, and a dectector is required at the end of the column to get a measurement; there are a wide array of different detectors that are commonly used, and they all have own settings to be dealt with. After the data is collected and the capsaicinoid peaks are identified, somehow the peak integrals have to be converted to concentration (ppm), and then that is converted to Scoville units. If all these things are the same, you will get statistically significant results (which can be confirmed statistically using an F-test.)
If you didn't specify a protocol, you were, in fact, just asking them to "do the best they can". You can have two world-class labs, with stellar instruments, staffed by the best technicians overseen by the smartest chemists, but unless they are running the same experiments, you will see a lot of daylight between their results. That's why experimental details matter, and why they should be in the reports you paid for. These aren't trivial issues, and labs (like mine) spend a lot of resources hashing these things out.
As other folks have already said, if you ever see values in a technical report that involve numerals reported with seven significant digits and no confidence interval, your spidey-senses should tingle. What they are REALLY telling you is this:
For a first pass, one measurement can be great -- especially for exploratory work, where you're really only interested in a qualitative or semi-quantitative value. All those extra digits look extra impressive and indicate that this just must be super-fantastic definitive work, but they should be interpreted as a sign of the exact opposite. Unless they acknowledge the limitation, it's a sign they have become accustomed to getting by with sloppy work in a culture of widespread innumeracy.
I'm not saying that you sent them to bad labs, but I am telling you that unless they run the same protocols, there is no reason on Earth to believe that they will get similar results. They need to use the same sample prep, same columns, same flow rates, same eluents, same sample loadings, same temperatures, same internal standards, etc. What's more, HPLC is only a separation technique, and a dectector is required at the end of the column to get a measurement; there are a wide array of different detectors that are commonly used, and they all have own settings to be dealt with. After the data is collected and the capsaicinoid peaks are identified, somehow the peak integrals have to be converted to concentration (ppm), and then that is converted to Scoville units. If all these things are the same, you will get statistically significant results (which can be confirmed statistically using an F-test.)
If you didn't specify a protocol, you were, in fact, just asking them to "do the best they can". You can have two world-class labs, with stellar instruments, staffed by the best technicians overseen by the smartest chemists, but unless they are running the same experiments, you will see a lot of daylight between their results. That's why experimental details matter, and why they should be in the reports you paid for. These aren't trivial issues, and labs (like mine) spend a lot of resources hashing these things out.
As other folks have already said, if you ever see values in a technical report that involve numerals reported with seven significant digits and no confidence interval, your spidey-senses should tingle. What they are REALLY telling you is this:
- they only did one measurement
- they don't understand how significant figures work, and don't think their readers do, either
- they have no real idea about how reproducible their measurement is
For a first pass, one measurement can be great -- especially for exploratory work, where you're really only interested in a qualitative or semi-quantitative value. All those extra digits look extra impressive and indicate that this just must be super-fantastic definitive work, but they should be interpreted as a sign of the exact opposite. Unless they acknowledge the limitation, it's a sign they have become accustomed to getting by with sloppy work in a culture of widespread innumeracy.
I don't think folks do understand the reliability of HPLC. Reliability is something that itself can actually be measured and quantified. In fact, for method validation and method transfer between labs, that's actually pretty important. I've seen no evidence that "long time members" have done the work to troubleshoot wherein the variability lies. I don't see any evidence that people are using "year after year" the same protocols under the same settings, but there seems to be plenty of reason to think that they haven't. If you spend money sending different samples to different labs who are running different techniques, and get different results, then congratulations: you're starting to learn why controlled validation is actually a really important value-added exercise.
Curiously, I still haven't seen the same keen level of scrutiny applied to the organoleptic protocol, a.k.a. Uncle Wilbur's pepper-squeezin' jamboree. Nobody is talking about the confidence intervals with that method, and I suspect it's for obvious reasons. If you have something that works better than HPLC, you have a bright future ahead of you in analytical chemistry.
Considering one of the labs was directly connected to the D.O.D., I have a slight feeling they know what they are doing. You seem to be a pep talker for the HPLC industry. There are many companies out there who have gotten very questionable results, from MANY different labs, for many years. Unless you know of a lab that rivals that of the Oracle of Delphi, I will stick to personal experience, and direct conversations with many others who have found the results questonable at best.
Well, one alternative is to not care about trying to quantify something seemingly for the sole purpose of hyping a product. I mean let's face it - if someone is that hung up on how many [X] units a sauce has, it's either for marketing purposes or because you're selling LTL aerosol systems to the defense dept.
So in my opinion the alternative is the age old tried and true "mild / med / hot / extra hot" rating. And if "extra" doesn't sufficiently describe the concoction and warn the user, the other marketing components do a pretty good job conveying the dangers....skull & crossbones, exploding thermometers, cartoon guy with pants down dipping buttocks in tub & generating steam, names like "DEATH" and "INSANITY" and "Ghost sauce", etc.
If someone buys one of these products and doesn't expect it to be hot, frankly they're probably not smart enough to understand either the Scoville or HPLC tests.
I think application is hugely significant. I get that the "super hot" sauce makers want a significant means of quantifying exactly how hot their sauce is as they seem to be competing for the claim to glory for marketing purposes. The military wants to know how hot their pepper spray is to ensure that it's effective - which is also a little silly because just like with tasting a sauce that's 1,002,340 versus a sauce that's 1,120,404 anything beyond a certain level ceases to have a material impact on the victim/subject. To paraphrase something Defcon eloquently said in an earlier post - really reay freakin hot is really really freakin hot.
IMO everything else is just marketing. It's good marketing in that it's working to educate the consumer - but it's also questionable marketing because the consumer who's drawn in based on whatever unit of measure also represents a smaller and smaller percentage of market share among all hot sauce consumers as that number increases.
I look at the best selling pepper sauces in the world - Tobasco, Tapatio, Sriracha - that audience would seem to represent the world's tastes for the majority. As a saucemaker I certainly want to appeal to the broadest consumer base. I may make an "extra hot" sauce down the road but I certainly won't launch with one. It'll cost more to make (more peppers), take the same lead time and presumably sell less.
I'm not going to disparage the companies that do brag about Scoville or HPLC ratings on their label/website/marketing materials...many of them are briliant marketers & collectors and hot heads keep product flying off the shelves. There is a definite niche there and with a seemingly never ending stream of limited editions & kitchy packaging these companies are very successful.
I just wonder how much repeat business they get from the average consumer, and what that interval is.
If not marketing or weaponization I just fail to see what the application of either scale is. If the latter, meh. If the former, show me a thermometer with a rough representation so I, the consumer, knows that I'm about to buy something edible or something that'll melt my face off. That's a totally acceptable alternative - unit ratings seem a bit like a pissing contest that nobody wins.
Sawyer
Extreme Member
Wow, go to sleep for a few hours and miss all the fun!
Having worked on some DoD-funded projects, I can assure you such a connection does not automatically imply infallibility. As you have discovered to your own chagrin. And from the experiences I'm hearing described here, "can" should be interpreted as "rarely, if ever". I'm still curious as to what you guys are paying for these tests.
I wouldn't necessarily say only hope; we're talking about detecting the concentrations of certain molecules in a matrix of other molecules. I can think of at least a couple of other approaches that might work, but apparently haven't been developed. So yes, for now, HPLC is the only practical hope. edit: And apparently not all that practical either.
I don't agree with your comments about protocols and think they only reinforce the negative opinions being aired here. I'm not trying to pick a fight here, but please consider this: I think we can all agree that a certain volume or mass of pepper, powder, or sauce contains certain concentrations of a number of different capsacinoids. If a particular lab cannot measure those concentrations within some reasonable degree of accuracy and have the results confirmed by a different lab using a different protocol, then either one or both of those protocols are fatally flawed. One of the hallmarks of good science is repeatability and reproducibility.
I directed an analytical lab for seventeen years (not HPLC but an alphabet soup of other techniques), so I know what you mean about a culture of innumeracy. It disappoints me greatly that this culture has made such inroads into fields that demand the most rigorous levels of numeracy. We can see the results right here in this thread.
Long time members shouldn't have to do the work to troubleshoot. If they submit a sample and pay the fee, they are entitled to expect valid results.
As you suggest, I think everyone recognizes the organoleptic tests are subjective and therefore don't expect the same level of experimental rigor. Hah, "Uncle Wilbur's pepper-squeezin' jamboree", you should trademark that.
PM me if you'd like to collaborate on a improved method. Invention's my game these days and I have the background. It sounds like you do, too.
As I indicated above, a DoD affiliation is no assurance of quality, so, no, it's not necessarily true they know what they are doing. And if they aren't providing reliable, quantitative results, then that just validates my point. I'm distressed to hear of the negative experiences of the users here and would like to know if anyone here has had a positive experience. HPLC not only can be a precise and accurate method of quantifying capsaicinoid concentrations, it should be such. If it's not then the aforementioned culture of innumeracy has spread further than I thought.
It's clear science is not serving society adequately here. This situation needs to be and can be fixed.
I agree that for consumable products, precise quantification isn't critical, but if measurement techniques can be refined to be more reliable, why not quantify? And yes, the military does need to know precise levels, at high enough concentrations, capsaicin can be lethal. But there are numerous other applications where precise measurement is critical as well. Pain creams, animal repellents, bear spray, self-defense spray, all these applications rely on precise compositional analysis.
I want to be clear here that I am talking about quantifying the capsaicinoid concentrations only. Converting those numbers to Scoville units is a whole 'nother ball of bee's wax.
Having worked on some DoD-funded projects, I can assure you such a connection does not automatically imply infallibility. As you have discovered to your own chagrin. And from the experiences I'm hearing described here, "can" should be interpreted as "rarely, if ever". I'm still curious as to what you guys are paying for these tests.
I wouldn't necessarily say only hope; we're talking about detecting the concentrations of certain molecules in a matrix of other molecules. I can think of at least a couple of other approaches that might work, but apparently haven't been developed. So yes, for now, HPLC is the only practical hope. edit: And apparently not all that practical either.
I don't agree with your comments about protocols and think they only reinforce the negative opinions being aired here. I'm not trying to pick a fight here, but please consider this: I think we can all agree that a certain volume or mass of pepper, powder, or sauce contains certain concentrations of a number of different capsacinoids. If a particular lab cannot measure those concentrations within some reasonable degree of accuracy and have the results confirmed by a different lab using a different protocol, then either one or both of those protocols are fatally flawed. One of the hallmarks of good science is repeatability and reproducibility.
I directed an analytical lab for seventeen years (not HPLC but an alphabet soup of other techniques), so I know what you mean about a culture of innumeracy. It disappoints me greatly that this culture has made such inroads into fields that demand the most rigorous levels of numeracy. We can see the results right here in this thread.
Long time members shouldn't have to do the work to troubleshoot. If they submit a sample and pay the fee, they are entitled to expect valid results.
As you suggest, I think everyone recognizes the organoleptic tests are subjective and therefore don't expect the same level of experimental rigor. Hah, "Uncle Wilbur's pepper-squeezin' jamboree", you should trademark that.
PM me if you'd like to collaborate on a improved method. Invention's my game these days and I have the background. It sounds like you do, too.
As I indicated above, a DoD affiliation is no assurance of quality, so, no, it's not necessarily true they know what they are doing. And if they aren't providing reliable, quantitative results, then that just validates my point. I'm distressed to hear of the negative experiences of the users here and would like to know if anyone here has had a positive experience. HPLC not only can be a precise and accurate method of quantifying capsaicinoid concentrations, it should be such. If it's not then the aforementioned culture of innumeracy has spread further than I thought.
It's clear science is not serving society adequately here. This situation needs to be and can be fixed.
I agree that for consumable products, precise quantification isn't critical, but if measurement techniques can be refined to be more reliable, why not quantify? And yes, the military does need to know precise levels, at high enough concentrations, capsaicin can be lethal. But there are numerous other applications where precise measurement is critical as well. Pain creams, animal repellents, bear spray, self-defense spray, all these applications rely on precise compositional analysis.
I want to be clear here that I am talking about quantifying the capsaicinoid concentrations only. Converting those numbers to Scoville units is a whole 'nother ball of bee's wax.
lol - that was my first thought.
I agree - when using as a weapon it's critical to be as precise as possibe.
When putting it on pizza, notsomuch.
Re: "uncle Wilbur's" - x2, that should be trademarked. Haha
Bottom line, if something is hot, it's hot. If something is friggin' hot, it's friggin' hot. Look at that, and it didn't cost a few hundred dollars to figure it out.
We've had literally thousands of people approach our booth at various shows, asking how many SHU's are in our stuff. My immediate reply is usually, "Why don't you tell me". I get a quizzical look, they try it, and the SHU answers I get from them are ALL over the map.
I will continue not to jump on the bandwagon of SHU's, and focus on flavor over heat. When it comes to making something hot, I'll just add peppers, and not worry about molecular density. What many people think is not hot, will blow the faces off other people. So, in fact, the scale doesn't really mean much to begin with. I do still love those thought challenged individuals that tell me they can tell the difference between 4.1 million SHU's and 4.2 mill SHU's.
We've had literally thousands of people approach our booth at various shows, asking how many SHU's are in our stuff. My immediate reply is usually, "Why don't you tell me". I get a quizzical look, they try it, and the SHU answers I get from them are ALL over the map.
I will continue not to jump on the bandwagon of SHU's, and focus on flavor over heat. When it comes to making something hot, I'll just add peppers, and not worry about molecular density. What many people think is not hot, will blow the faces off other people. So, in fact, the scale doesn't really mean much to begin with. I do still love those thought challenged individuals that tell me they can tell the difference between 4.1 million SHU's and 4.2 mill SHU's.
Sawyer
Extreme Member
You put pepper spray on your pizza!? That's heavy duty right there. My hiking buddy and I once sprayed bear spray in our rehydrated meal packs. That was interesting. The grain alcohol we carried as "fuel" might have had something to do with it.
Hahaha - uhm, no - but I was able to taste pepper spray in a controlled setting. Surprisingly not as hot as some extreme hot sauces I've tasted & it was really really really disgusting flavor-wise.
ETA(controlled = friend's father in law is a chief of highway patrol (ret) & busted some out at a BBQ when we got to talking about hot sauce...I do not recommend agitating a cop or goading them into spraying you. I hear it's really a different & unpleasant experience. lol)
But you also validate my point - if I'm not gonna use pepper spray on my pizza then why do I really care *precisely* how hot it is? The thermometer image works just fine & doesn't have any numbers on it.
I've been considering this aspect to marketing & I'll likely do the thermometer. I considered a plastic lid overwrap thing that says "hot" "med" etc on it, but that thing gets peeled off. I guess I'll get 10 friends together & do a taste/heat test - ill give 'em each a couple of beers, a thermometer picture & a crayon & tell them to mark how hot it is. The average will be what I go with.
Seems scientific enough for pizza?
Sawyer
Extreme Member
Absolutely. I was completely unaware of this problem with HPLC until I came into this forum. Unless and until these inconsistencies are ironed out, it seems pointless to quote relatively meaningless numbers on a consumable product.
Somewhat on topic, I've started working on a low-cost kit that folks such as yourself could use to conduct your own
I'm in as a volunteer if it'll help!
Bring on the Pepper Squeezin' Jamboree!
Bring on the Pepper Squeezin' Jamboree!
There are times where you really need to know how pungent something is, and I agree that pepper sauce probably isn't the top priority. A lot of folks in my lab used to measure capsaicin concentration in a QC lab setting. The product? It was a topical capsaicin cream, and so it was considered an active pharmaceutical ingredient (which it honestly was), and so following FDA guidelines, they needed to develop and validate a method. To meet FDA requirements, they used HPLC to measure the content; it worked. By contrast, they didn't sit around sipping diluted solutions of their medicated cream, asking each other, "Hot enough for ya?... How about now?"
This is where it gets a little tricky, especially with some of the marketing for newer pepper sauces. The bottles with skulls and flames and colorful warning signs are a little hard to gague. Is "will make you go blind" hotter than "will make you fart blood"? How does "you will forget how to tie your shoes" compare with "renal system shutdown" or "will deposit a nest of scorpion hatchlings in your esophagus"? Which sauce is right for me?
A more quantitative measurement would be appreciated, but we're just not there yet.
True enough. Regardless, as a scientist, it frosts my weenie when I hear bad chemistry bandied about. Saying that HPLC doesn't work/isn't "reliable", with no discussion of how the testing was done, or what the presumed reliability acceptance criteria are is just not something I can quietly abide. This kind of misinformation has a tendency to linger for years or decades as "common wisdom" and doesn't help a thing, and it takes a huge amount of effort to clean up.
Robust methods for HPLC analysis are developed and used every single day on things that are far trickier to measure than this, and so it should come as no surprise that there are analytical standards in place for capsaicin. There are thousands of peer-reviewed published articles out there where the analysts use this technique (which is a good thing), and so there are many, many competing protocols out there (which makes things a little messy.) It's considered an active pharmaceutical ingredient, and as a consequence, the U.S. Pharmacopeia's monograph on capsaicin is a good place to start. In the space of a few minutes, I was able to get almost two dozen articles on hot sauce analysis alone, with some big differences in technique.
This is where it gets a little tricky, especially with some of the marketing for newer pepper sauces. The bottles with skulls and flames and colorful warning signs are a little hard to gague. Is "will make you go blind" hotter than "will make you fart blood"? How does "you will forget how to tie your shoes" compare with "renal system shutdown" or "will deposit a nest of scorpion hatchlings in your esophagus"? Which sauce is right for me?
A more quantitative measurement would be appreciated, but we're just not there yet.
Yeah - I hear ya. My favorite was the review for a sauce (forget the brand) that under taste it said, "it tasted like a bee sting in my mouth"
While I agree from that scientific perspective it's interesting, when it comes to eating the stuff there's seemingly as much variation in the human factor as there is in the capsaicin content. People's inherent and built up tolerance ranges from folks who can't handle a bell pepper to those who chow down Red Savinas and laugh in your face, mocking your girlie taste buds.
Yeah - I hear ya. My favorite was the review for a sauce (forget the brand) that under taste it said, "it tasted like a bee sting in my mouth"
While I agree from that scientific perspective it's interesting, when it comes to eating the stuff there's seemingly as much variation in the human factor as there is in the capsaicin content. People's inherent and built up tolerance ranges from folks who can't handle a bell pepper to those who chow down Red Savinas and laugh in your face, mocking your girlie taste buds.
In my hometown of St. Paul, there is a tremendous demographic shift going on. Minnesota has a substantial population of Scandinavian Americans, but in the last 30 years, there has been a major influx of Southeast Asians -- in particular, Cambodians, Laotians and Hmong people. What used to be considered "spicy" (e.g. Heinz ketchup) isn't so "spicy" anymore. When I go to a Laotian restaurant and order something, I've learned I not ask for "hot"; since they look at my European-looking face and think I don't know what "hot" is. I learned its important to make strong eye contact and specify, "make it Laotian hot. As in, make it hot for Laotian people". This usually works.
