yes, i think it makes a difference. It's best used when you have organic microbiology to help bring the trace elements to the roots, so use things such as mycorrhizae, worm / compost tea. It's best if you have raised beds or something you can mix in with compost, but I put it in my potted plants. Another option that I like to use regarding trace elements is a liquid version, called ocean solution. not only does it have trace elements, but it has beneficial bacteria in it too. You can use is when watering or use it as a foliar spray.
if you want cheap rock dust, not just azomite, boogie brew teamed up with john kohler of growingyorugreens youtube channel and I think I saw a video where he's selling it super cheap. Something like boogiebrew.net/gyg