I grow milkweeds mostly for the butterflies (Monarch Butterfly). By the end of the summer my milkweeds are just stubby branches sticking out of pots! They grow back eventually. Most butterfly larvae are host specific, so you shouldn't worry about the larvae on the milkweed eating your peppers, even if they crawl onto the pepper plants. I am surprised you don't have any wasps coming and taking the caterpillars away?