First off, you can go and have a look around without being a member, or being with a member.
It is worth it once you have your set things, and have compared prices. Pasta is more expensive at Costco than Coles.
We always buy from there: cat litter, dishwasher tablets, kitchen towel, tubs of pre cooked rice, bacon, frozen chicken wings, orange juice, tubs of mandarin segments, tins of tomatoes, razors, deodorant, laughing cow cheese, shredded cheese, half lamb packs.
We did buy lots of things at the start that you realise later you don't actually use all of within the use by dates. So now we stick to those things. I will buy kids books there sometimes too.
Before I was on maternity leave I would just stop off on the way home from work once every 3 weeks or so. I'm going less often now as its a bit out of the way and they don't have baby trollies.
But go and have a look around. Have the Coles online shopping open on your phone so you can compare prices.
One great thing with Costco, if you buy anything there, and keep your membership up then it has a lifetime guarantee. Lots of people on their Facebook fan page thingy have said they have taken things like TVs back 3 years later and got a refund with none of the having to send it away with other places etc. however, their electrical items more more on the higher end.