Bummer about the nosy neighbors. None of their business what you do in your own home, (as long as you're not driving property values down ) I'm surprised that we don't see more posts with the same subject.
I was thinking about that happening to me just recently and was wondering what I would do. I decideded as long as they limit their search to the pepper grow room I wouldn't mind.
Once you let them in the door, you have already invited them in and given up your right to unreasonable search and seizure. Personally, I would never let the police in my house without a warrant.