I wanted to do it for years and always found reasons why it couldn't happen. I wish I would have started doing track days so much sooner. Its the most fun I have had on motorcycles in years. Plus the community aspect of it is really cool. Your surrounded by a bunch of like minded people that won't roll their eyes when you talk about the nuances of suspension design.
It doesn't need to be a well planned out and expensive endeavor, you can rent typically rent suits, bikes and whatever you need from track-side vendors if you just want to dip your toe in before fully committing. My only advice would be just get out there and do it!