Rushmoor Council's executive cabinet was accused of being "totally out of touch" following its decision, which it is feared will have a severe impact on local businesses when it comes into effect in September.
The cabinet decided to introduce the charge despite opposition from many councillors of all parties.
The Tory cabinet decided to introduce the charge on Tuesday — the day after a council scrutiny panel called in the proposals for reconsideration.
The move has angered David May, chairman of the Aldershot Town Centre Management company, who accused the cabinet of being totally out of touch and making a retrograde step on the future of the town.
He said: "They are only interested in getting the revenues they need, but they will not even get that because people will just stop coming."
He said free parking had been the main marketing tool to attract customers who would otherwise go to centres like Guildford and Camberley
He added: "The effect will be to reduce the amount of takings for commercial enterprises who may then move out.
"This comes at a time when it is not an easy trading situation anywhere — it may weaken us to the point where we can not recover."
These views were echoed by Gary Little, manager of the Princesmead Shopping Centre, who said the redevelopment of Farnborough and traffic problems had already had a severe impact on the town centre.
He added: "Farnborough is seriously in decline.
The proposed charges could just cement people's already jaded perception of Farnborough and make them travel further afield to Guildford, Camberley or Woking.
"While we appreciate they will still have to pay for parking, the retail on offer in other areas is far superior to that of Farnborough."
Cllr Keith Dibble, the leader of Rushmoor's Labour Group who called the issue before the scrutiny panel, said: "At the moment there is a piecemeal approach to setting charges which is causing confusion.
"This was a lost opportunity to take car parking out of the political battlefield."
He said a proper review needed to be undertaken which involved traders, so that fair and workable charging could be achieved.
At Monday's scrutiny panel meeting, councillors were spilt on whether to reject Sunday charging and the abolition of the concessionary "free after three" scheme in Aldershot.
Tory Cllr John Wall refused to give the casting vote on whether to reject the cabinet's new charging regime or defer a decision.
Despite opposition in recent weeks, the cabinet on Tuesday unanimously agreed to press on with imposing the charge, which will cost Sunday shoppers 40p whether they stay for an hour or the day.
Cabinet member responsible for parking Cllr Roland Dibbs, who attended the scrutiny committee's meeting to answer questions, said: "At the moment there are no parking services on Sundays but when the charging comes in, the lifts will be put into operation and there will be someone patrolling the car parks."
In response to claims by some councillors that the consultation and decision process had been too hasty, deputy council leader Cllr Peter Moyle argued: "Last year we said we were going to defer it to this year so I am amazed anyone was surprised.
"Consultation is a two-way exercise.
"People are not coming back to us but going to the press and saying this is being foisted upon them by the cabinet."
He said traders were consulted but said a lack of response had been disappointing.
Last week Tory Cllr Roger Kimber was one of a number of councillors who attacked the cabinet.
He said: "The shopping economy is fragile and this will only add to the problems.
"It is a very serious and emotive point at the moment."