Many Brickfilmers use a range of software, for instance for frame capture many Brickfilmers go with MonkeyJam, which is free, though there are many other frame capture software such as Stop Motion Pro and AnimatorHD, they cost money though. You are right to use Sony Vegas for editing, it’s a great professional program and it’s also what I use. For recording and editing audio Audacity is a popular choice, it’s also free.

I use Stop Motion Pro v7.5, Action! HD version.
It has a lot of features, and I love it. It’s 185$.

I just use iMovie and iPhoto, sure iMovie isn’t even on the top 50 programs to use, for these types of films but, it works for me. All my videos on YouTube are made using iMovie. Also, I just take pictures with my camer and upload them to iPhoto, drag them into iMovie, and I somtimes use garage band for audio, though I’m not sure exacly how to use garage band to its full extent quite yet.

anybody know a free animation software for mac?

I use monkey jam as well. It’s a very decent software and easy to use. It’s free of course so it helped me in a learn a lot of things through it. For advanced software I use AP. I’m just not really sure though if that one is considered to be advanced now.