Yes, no-one else is in a similar dilemma of course...
well the high street is full of pretty revolting stuff in my opinion these days - plus, with mass production there is a lot of the same stuff around...
as far as ethics are concerned, some shops are definately worse than others - GAP, as you say, are known offenders...you need to find out where the products are made or outsourced to have any clue as to production practices...
You could always buy a lot of your clothing at festies - like your (ahem) 'leisure wear'...then you know that your money is only going to hippies...there was a hemp clothing woman at the Glade I had a long chat with and she also wanted to go over to ethically traded hemp from Nepal - fair trade cloth/fabric...
You could get someone to make up your clothes - there are loads of tailors where I live and they're around, and there's NOTHING like a wellcut pair of trousers ("Oh, suit you Sir")
Faeriebee is also right, if you can be bothered to google...
But as far as workclothes are concerned the cheaper the clothing is the more likely it is a sweatshop product - that's why handmade/craftwork stuff is always so expensive - you're paying for someone's time and skill...
OH yeah - there was an interesting article in the paper a few months back where the journo traced a shirt given to a charity shop - when it didn't sell it was shipped to Africa where the rag trade takes it to small markets in places like Mozambique. It was bought by a young woman and the moral of the tale is I guess while the revolution is that far off you might as well buy clothing regularly, donate your old while it is in a very good condition and then the poorest people who buy it last will get something nice - good quality recycling.
At the end of the day, the rag trade should be one of the cleanest trades on the planet - nearly everyone likes good clothes and it should be sustainable too - nature grows the seed etc...
Sorry not to have been much help
Mind the GAP
Meijin