jasongd, everyone is going to have differing views on what places are like depending on their own expectations and experiences, likes, dislikes and how they live their lives. Not for everyone is a bustling capital city or large coastal city with a big tourist industry and so on.
From what you've said, I'd think Perth or Adelaide are more suited to your needs and preferences. And if you want slower pace of life, more small town then perhaps SA near Adelaide is more suited to you.
With regards to the comment made by your mother, times have changed and I think to disregard an entire state/city area based on her opinion is a bit daft. I've spent plenty of time in SA in and around Adelaide and have never felt unsafe or worried about my personal safety. My husband lived near the CBD for a while and when I was there with him, even then I felt safe and didn't worry overly about city living and all it might entail. But like everywhere, things can happen. Even in the smallest, quietest village in rural England a murder, car being stolen or robbery can occur. Same can happen in Australia. Or the US, Canada, France.....
I'd make a list of what you want from a move and then do some more research into both cities/areas.
I personally love Adelaide. It is a slower pace of life, has some lovely beaches that are far removed from tourist filled or over used and some small towns outside of the city that might tick your boxes. Or even a suburb of the city as it sprawls so far now. My husband is from there so it helps us that returning there is a given. But I'd not move there if I didn't think it was a good place to bring up our son (now 3) or didn't have what we needed and wanted for our lives. I have no qualms about moving to Adelaide and I have done loads of reading, research and spent plenty of time there also which helps. I certainly didn't want to be in one of the bigger cities or living a faster pace of life. For us that is not what its all about.
Also the climate is decent. It can get very hot in summer (40c) plus and its a dry heat, not a humid or tropical heat. It does also get colder in winter but not cold as in UK cold. I've yet to spend a winter in Aus, only spent spring, summer and autumn so far but loved the spring and autumn, found the summer a bit hard going but only because I hadn't fully adjusted to 30C plus for days at a time. However, I know I'll be fine once living in it as you do adjust to the climate you live in.
I can't comment on Perth as have not been there. I know during my initital research I did rule it out for us mainly because it did feel like it was more isolated and away from things. I know its only a plane ride to elsewhere but I felt that along the coast, there was only so far you could go before reaching limits and also inland. I wanted to be somewhere there was 'more'. Being able to drive either way, reach Melbourne in a day, have other options with regard to trips, days out and so on. I felt we would exhaust this quicker in Perth than anywhere else major city wise in Australia. Of course the family ties with Adelaide were a big factor also. But Perth never felt in the running for us. Melbourne was an option but again, we don't need busy nightlife, trendy bars or anything to do with that bigger city living. Also it was far more expensive than Adelaide. End of the day for us family ties, love of the city and surrounding area, schools, areas to live, those were the reasons we decided to return to Adelaide over somewhere else in Australia. Both hubby and I are happy with this choice and are working toward our return :)
End of the day, what your requirements for a move are will ultimately lead you to your destination. Do plenty of reading and remember everyone will have an opinion Take a look at Flickr for albums from both areas, loads of people post their photos publicly and you can see more of what each area looks like. Or use Google images or something. Also do lots of reading, tourist sites, schooling, areas to live, prices of accommodation, jobs and more. It'll all help give you a better idea.