It is horrible making the jump from a 'normal' diet to gluten free. It can be more expensive, but doesn't have to be!
The problem is that the foods you think of as nice, or a treat, or junk, are far more likely to contain gluten than anything else, mainly because flour and grains generally are cheap fillers... What you need to start eating are basic foods like meats, cheese, potatoes, dairy, rice, veg and fruit. Processed foods are a nightmare - the ingredients list is tiny and complex!!
I eat GF easily now I'm use to it. I cook nearly every meal (you might as well learn as you go, you'll be glad you did), and buy the odd GF ready-made food as a treat.
GF packaged food varies hugely. Stay away from the awful muck the dieticians give you vouchers for, most of it is horrible! The things I buy again and again are supermarket GF pasta, etc. The better the supermarket, the better the quality and taste, I've found. Waitrose are great at GF and their prices stack up very well against any other GF brand. They make treat type things like cherry bakewells, muffins, crumpets etc very well. The other really good make is DS, their pasta, pitta breads, crumpets etc are the best I've found.
Just make sensible swaps in your diet - forget bread and get over the loss thing. It's a waste of time and energy pining after stuff that makes you ill. GF pittas work really well, and make a good pizza base too!
An important point is that there's no point AT ALL in making the change if your kitchen is contaminated with gluten everywhere. You must keep your storage area, work surface, pots and pans, etc scrupulously clean if there's gluten around. Use a separate cupboard, chopping board, grillpan, etc. Never use the same toaster as a gluten-lover!
It's soooooo worth it. This all seems hard work, but you get used to it quickly, and you'll feel better. Strangely, I'm spending LESS now because I'm not buying processed food, and my kids are healthier for it.