The 2M TSMB: Interpreting the results

[pepperjoe]

MRZ1988, Nice job of breaking it down into laymans terms and a good common sense explanation.
I think the key number is obviously the 'Mean Heat' number.
Those swings from the High Heat to the low heat are disconcerting at the very least.
Bhut Jolokia low heat 279,000....wow.
There's been far too much press on the High Heat of the Moruga Scorpion at 2,000,009 which was obviously a one-off.
I wonder what those numbers would look like if you sampled say 8 plants instead of 4, then took off the Highest and lowest number...and then re-calculated a Mean number?
Hmmmmm....
My guess is the Jolokia would look better, the Moruga would look worse.

 

[synclinorium]

MRZ1988, Nice job of breaking it down into laymans terms and a good common sense explanation.
I think the key number is obviously the 'Mean Heat' number.
Those swings from the High Heat to the low heat are disconcerting at the very least.
Bhut Jolokia low heat 279,000....wow.
There's been far too much press on the High Heat of the Moruga Scorpion at 2,000,009 which was obviously a one-off.
I wonder what those numbers would look like if you took off the Highest and lowest number...and then re-calculated a Mean number?
HMmmmmm....

If nothing else, this study tells us nature is fickle. We may be a long way off from truly standardizing the heat in the superhots, if ever.

Then again, this may just be typical of peppers. Something as basic as a jalapeno to demonstrate my point: most of the jalapenos I've had in my life have been relatively mild, some with almost no heat, but occasionally I'll run into one that is downright nasty. Are we dealing with a non-normal distribution here? If so, the mean would give an inflated estimate of the typical pepper, as most of the data would be clustered toward the lower end up the spectrum.

Averaging the value over several plants helps to reduce the influence of outliers, but even so, at four plants per sample, that may not be enough.

 

[mrz1988]

Thanks for the feedback guys. Frankly I agree that testing for this kind of stuff in a lab is relatively useless. Nature is just too unpredictable in things like this and these studies are nothing more than an attempt to put a number on something that probably doesn't need a number to begin with. It really only comes down to marketing. The Bhut Jolokia was a clear exception in its time because it could be confirmed as hotter than the Red Savina on all counts by a large margin. Today there are too many strains at the top for us to try to say which one is #1.

A couple clarifications on strain have been made since I wrote this post so I thought I'd share them:

-The 7 Pot used was said to be the Jonah strain
-The Chocolate 7 Pot used was a Douglah that was renamed for the study because of its incredibly close genetic similarities to the Jonah
-The Scorpion used MAY have been a Butch T but it was not from the same batch of seed that set the record last year

The more I think about it the more I think that studying this fully would require more data points than is financially feasible. It might even be better to collect data points over several years and grow the same plants each year to try to eliminate changes brought on by growing conditions.


The bottom line is that the hype is just hype and if you want an insanely hot pepper, grow any of these varieties. The rest is just novelty brought on by marketing masquerading as science.

 

[xcphantom]

I say i would rather hear the the results from the test designed by wilbur schoville then hear the results from the hplc test i know both are "accurate"
but i think it would be better to do the schoville test since its cheaper (its much cheaper to spend 15 bucks for 5 college kids to tell you if a pepper is hot or not then spending 10000 bucks for a machine)


http://ushotstuff.com/chileFAQ.htm
The Scoville Organoleptic Test is a refined, systematic approach. With this method, human subjects taste a chile sample and record its heat level. Samples are then diluted until heat can no longer be detected by the taster, this dilution is called the Scoville Heat Unit, named for the man who invented it, Wilbur Scoville. A more technologically advanced test is an HPLC test, or High Performance Liquid Chromatography. An HPLC ‘sees' the heat compounds and records the amount in parts per million (ppm). A quick conversion from HPLC to Scoville is to multiply the ppm by 15 to get the Scoville Heat Unit.

 

[j.t.delaney]

Thanks for the feedback guys. Frankly I agree that testing for this kind of stuff in a lab is relatively useless. Nature is just too unpredictable in things like this and these studies are nothing more than an attempt to put a number on something that probably doesn't need a number to begin with. It really only comes down to marketing. The Bhut Jolokia was a clear exception in its time because it could be confirmed as hotter than the Red Savina on all counts by a large margin. Today there are too many strains at the top for us to try to say which one is #1.

A couple clarifications on strain have been made since I wrote this post so I thought I'd share them:

-The 7 Pot used was said to be the Jonah strain
-The Chocolate 7 Pot used was a Douglah that was renamed for the study because of its incredibly close genetic similarities to the Jonah
-The Scorpion used MAY have been a Butch T but it was not from the same batch of seed that set the record last year

The more I think about it the more I think that studying this fully would require more data points than is financially feasible. It might even be better to collect data points over several years and grow the same plants each year to try to eliminate changes brought on by growing conditions.


The bottom line is that the hype is just hype and if you want an insanely hot pepper, grow any of these varieties. The rest is just novelty brought on by marketing masquerading as science.

I guess I'll play the role of optimist for once. The interesting thing here is that it isn't so easy to answer the seemingly straightforward question "what is the hottest pepper in the world?". It's the inherently tricky thing with handling the superlatives for biologicals, isn't it? Sure, the hottest measurement of any pepper may currently be for Moruga Scorpion, but that's not even half the story. If we assume that capsaicin expression follows a normal function, we can expect that plotting out the values for each population of peppers will follow something like a Poisson distribution, with apparently very heavy overlap between the different superhots.

There is actually a lot of interesting, practical questions embedded in this, when you think about it. Trinidad Scorpion seems to be able to really deliver great mega-SHU consistency -- so why exactly is there so much scatter in the Moruga Scorpion samples? It would be great to see more datapoints -- both to gain an appreciation for the repeatability of the test, but also to see how well behaved our datasets really are. Is the Moruga Scorpion data truly monomodally distributed, or could we be looking at polydispersity? Could selective breeding within the Moruga Scorpion population eliminate some of the dispersity and bring up its mean?

I say i would rather hear the the results from the test designed by wilbur schoville then hear the results from the hplc test i know both are "accurate"
but i think it would be better to do the schoville test since its cheaper (its much cheaper to spend 15 bucks for 5 college kids to tell you if a pepper is hot or not then spending 10000 bucks for a machine)


http://ushotstuff.com/chileFAQ.htm
The Scoville Organoleptic Test is a refined, systematic approach. With this method, human subjects taste a chile sample and record its heat level. Samples are then diluted until heat can no longer be detected by the taster, this dilution is called the Scoville Heat Unit, named for the man who invented it, Wilbur Scoville. A more technologically advanced test is an HPLC test, or High Performance Liquid Chromatography. An HPLC ‘sees' the heat compounds and records the amount in parts per million (ppm). A quick conversion from HPLC to Scoville is to multiply the ppm by 15 to get the Scoville Heat Unit.

If you have the tools in an academic lab, this kind of work can be done on the cheap; undergrad lab assistants are easy to fund, and autosamplers mean that samples can be run 24/7. I would contend that the Scoville organoleptic test is actually much, much more expensive and much less reproducible: you will burn through college volunteers far faster than chromatography columns (especially if you're expecting to pay five of them 15 bucks -- three dollars doesn't go very far these days). There's a reason that you won't see standard deviations expressed in those measurements. The good Dr. Wilbur lived and worked in an era before statistical analysis (not to mention chromatography) was applied the way it is now; he did the best with what he had available. Since the 1980's; there have been thousands of peer reviewed papers reporting the quantitative analysis of capsaicin, only a handful of those use organoleptic testing, and all of those papers are from Pakistan or China.

 

[mrz1988]

I guess I'll play the role of optimist for once. The interesting thing here is that it isn't so easy to answer the seemingly straightforward question "what is the hottest pepper in the world?". It's the inherently tricky thing with handling the superlatives for biologicals, isn't it? Sure, the hottest measurement of any pepper may currently be for Moruga Scorpion, but that's not even half the story. If we assume that capsaicin expression follows a normal function, we can expect that plotting out the values for each population of peppers will follow something like a Poisson distribution, with apparently very heavy overlap between the different superhots.

There is actually a lot of interesting, practical questions embedded in this, when you think about it. Trinidad Scorpion seems to be able to really deliver great mega-SHU consistency -- so why exactly is there so much scatter in the Moruga Scorpion samples? It would be great to see more datapoints -- both to gain an appreciation for the repeatability of the test, but also to see how well behaved our datasets really are. Is the Moruga Scorpion data truly monomodally distributed, or could we be looking at polydispersity? Could selective breeding within the Moruga Scorpion population eliminate some of the dispersity and bring up its mean?



If you have the tools in an academic lab, this kind of work can be done on the cheap; undergrad lab assistants are easy to fund, and autosamplers mean that samples can be run 24/7. I would contend that the Scoville organoleptic test is actually much, much more expensive and much less reproducible: you will burn through college volunteers far faster than chromatography columns (especially if you're expecting to pay five of them 15 bucks -- three dollars doesn't go very far these days). There's a reason that you won't see standard deviations expressed in those measurements. The good Dr. Wilbur lived and worked in an era before statistical analysis (not to mention chromatography) was applied the way it is now; he did the best with what he had available. Since the 1980's; there have been thousands of peer reviewed papers reporting the quantitative analysis of capsaicin, only a handful of those use organoleptic testing, and all of those papers are from Pakistan or China.

This gives us a lot of interesting things to think about. Once again thank you. Statistically speaking, I hadn't even thought of the possibility that pepper heat isn't always monomodal. I'd like to think that particularly well-stabilized strains would be. There is the possibility that there is a gene expression that would cause two or more different heat levels to be expressed from the same strain. What if the TSMB is in fact polymodal and that explains its wide standard deviation? Could it be that it's still a tad unstable as far as its heat genetics go?

Like we've expressed, this is all doable through experimentation but we would need to have one hell of a dedicated lab to do it. I think that the primary goal of this experiment was not to produce a scientific conclusion as far as the hottest in the world, but rather to spark more debate. More importantly, the CPI needed to get more attention by winning the title of hottest pepper once again using their testing. I think it serves to hold some of the community to a higher standard for testing (using multiple samples, labs), but still not high enough.

It's too hard for small-time breeders to go test 100 samples of the same strain. Heck, it's hard for the CPI to do it evidently... they just haven't been held to enough peer scrutiny yet to be encouraged to test more. In reality its like they took five six-sided dice and tried to figure out which one rolls higher numbers by rolling each one twice. It should leave true scientists and statisticians with their palm firmly planted on their forehead.

 

[xcphantom]

I have a question? did they ever see if the HPLC test was ever 100% acurate?

 

[j.t.delaney]

I have a question? did they ever see if the HPLC test was ever 100% acurate?

No quant analysis is 100% accurate. The best you can reasonably hope for is demonstrated, documented accuracy within reasonable acceptance limits, which is a key part of method validation. To the best of my knowledge, they didn't follow a validated protocol, but then again, that would make this into a much bigger, more extensive production. Then again, anybody who would publish a complete method validation for a facile, (locally) optimized capsaicin from peppers would be pretty much guaranteed a huge number of citations, since people would simply cite the so-and-so protocol in their experimental section. Hm...

 

[Maxsack331]

No quant analysis is 100% accurate. The best you can reasonably hope for is demonstrated, documented accuracy within reasonable acceptance limits, which is a key part of method validation. To the best of my knowledge, they didn't follow a validated protocol, but then again, that would make this into a much bigger, more extensive production. Then again, anybody who would publish a complete method validation for a facile, (locally) optimized capsaicin from peppers would be pretty much guaranteed a huge number of citations, since people would simply cite the so-and-so protocol in their experimental section. Hm...

hmm, is right, lol.... good way to become relatively well known

 

[xcphantom]

wow and i have a A in bio O_O i could barely understand that lol its nice to know that there are a few scientists on this site

 

[Jay]

Alright so I'm rather new to this forum, but one of the reasons i came here was to discuss this very topic. I would like to know which one is "the hottest"

Right now I'm a masters student at the University of Waterloo, and have access to an HPLC for the price of $19/hour. Each run would take approximately 10 minutes, so I would guess at 6 tests an hour. So $3 per sample for me to run the HPLC personally, which I have done many times before when I worked in the analytical instrumentation lab.

This is about as cheap as its going to get.

If I had the space/ seeds/ time I would most likely attempt to grow many of the plants myself, however based on the previous replies, the required sample size for "acceptable results" is very large.

Do you think enough people on this forum would be able to grow peppers and submit them dried, while strictly controlling growing conditions to monitor stress on the plant? If we split up the growing between many people, and assigned growing conditions to control (ie. temperature, watering, ferts etc.), then perhaps we would have a large enough sample base to get results that everyone would be happy with. I would gladly distribute all data publicly on the forum, including all individual results of peppers.

The disadvantage to this is that it would all be tested in one lab on the same machine (but hey maybe that's a good thing). I am in no way connected to the chile industry, and for the sake of unbiased testing I would request that anyone who grew peppers for a test such as this would also be unbiased.

What does everyone think?

 

[meinchoh]

from Ohio, Jay!! You may become quite a busy young man!

 

[nitwit]

so we are "interpreting the results"

i would think the results need to be: published, released, confirmed, or otherwise
distributed before any interpretation is made (there was a periodical mentioned by the researching group?)

the $3 hplc test sounds groovy (who do you serve? and who do you trust?)

but first it may be wise to wait and to gather information before scrutiny

it may be another sample will be tested for the knobbly type or smooth type even

by the (any) others
before any publishing or release of data another may rate even (ever) higher?

there is a need to let the researchers give out information as they chose to (or not to)
before demands of proof are even "valid"

i am as impatient as the next one for all the information
but it was not my $3 spent for the information that the (others) above chart related to

they need to control their "investments" in this before 'proving' to any of "them"

 

[Maxsack331]

from Ohio, Jay!! You may become quite a busy young man!

lol yeah I second that... I would say if you could wait another 2 months you would have plenty of samples.. and hell if you get enough you could easily just combine some or I dono do something to make it easier and get a better "average"... I guess you could either do it so that it is very specific like the original test, or you could go for a more general test with a "handful" of peppers of the same time from each source and test that.. so you get a few "clumps of peppers" from different parts for each type of superhot.. but something like that...

I would keep doing some research and in about a month, post on here about collecting dried peppers and $3 for each "batch" (superhot contenders only) and preferably no crosses or whatever (unless of course you could find one that is crazy hot like the brain strain, satan's strain.. like Dale has from Baker's Peppers, although I can't remember who he originally got them from, there was a thread on here about it)... would be pretty cool and very interesting to see the results.. hell if you were willing, I'm sure a lot of people would just send a few of their superhot peppers just to test to see how hot they really are, for personal reasons

just need to see if you can isolate the capsaicin yourself and effectively, I can't remember if they discussed it in this thread, I gave up trying to read it all, lol, i have enough of my own bio stuff to read and remember right now for classes

 

[Jay]

I'll spend the next week trying to isolate capsaicin. I'm in chemical engineering though, so I would hope I would be able to do it. If not i might need to go back to undergrad .

In terms of who i serve and who i trust; I would hope I can trust the people on this forum to be genuine about what they send. And I don't serve anyone in particular... I'm just very curious and would hope to maybe clarify these heat ratings, and do the tests/sampling how everyone else would like.

 

[Maxsack331]

yeah, I figured, I just don't know if they said how to do it, and it's been a while since I've thought about chemistry, I personally like biology more

but I think Nitwit was joking about that statement about who do you serve and who do you trust.. but if not then I don't know what he meant by that lol.... you will find the people on this site great! I joined a few months ago and have learned TONS of stuff.. and people are much more willing to help and give input than other forums that I have seen

 

[xcphantom]

OMG ITS AN INVASION OF SCIENTISTS lol

Welcome to the site Jay hope you enjoy it here (I'm a freshman in high school and I want to go into botany)

 

[mrz1988]

Hey Jay, I'd love to be a part of your experiments, it could be fun. The problem with finding 'the hottest' is it's just too hard to do without a ton of data points. There are at least a couple dozen varieties at the top and more come out every year. Testing them all would require an enormous controlled garden, otherwise results won't have a very high level of consistency. We can find the hottest in a given sample size, but the same results will not be able to be replicated in another garden or another year consistently. Every grow and every plant will create different heat levels, and some growers will create consistently hotter chiles than another. Of course, the more data points we have, the better idea we will have on how hot these chiles really are. Definitely keep me posted it sounds like you have some cool things going on.

 

[Maxsack331]

I want to see who has the hotter peppers around here.. then mob them for seeds haha.. and it would be cool to be able to tell people.. well brag to people, "how hot my pepper is" lol.. assuming I get some nice peppers

OMG ITS AN INVASION OF SCIENTISTS lol

Welcome to the site Jay hope you enjoy it here (I'm a freshman in high school and I want to go into botany)

haha, nice, I graduated a year and a half ago with a BS in bio. took a break, was planning on finding a job with animals.. anything really as long as it wasn't lab work.. couldn't find much that I would want, so now I'm taking classes again for prerequisites for physician assistant school... I just wish I had found out about growing hot peppers when I was in school, I could have been all over their green house.. lol, would have a bunch of superhots in there, and I could have had 4 years to come up with some cool crosses.. my mom has always had a garden, but I never cared a whole lot about it.. until I could get hot peppers to actually grow here!.. it's so much more fun when you get trick people into eating pain haha

well back to a little studying.. don't think I will get too far tonight

 

[Jay]

I like the idea of starting a friendly competition to see who can grow the hottest pepper! Could be a lot of fun.