The travellers started moving on to the Peter Driver playing fields at lunchtime on July 16 and by the evening there were about 70 vans and caravans.
Worried residents immediately called the police and Hart Council, which owns the land.
One said: "I understand it's a group from Southampton who were moved from a site by the M3 at Cove.
"The police said that wherever this group goes there is a mini crimewave.
"I've seen them riding motorbikes all over the football pitches, so they will be ruined."
Crookham Krakatoa football club has also been left wondering if the pitches will be playable for the start of the season.
One player said: "The frustrating thing is we lost a number of matches at Kennels Lane when the travellers invaded there.
"It looks as though we could have the same problem here.
"We pay for those pitches and want to play football. It's really frustrating that two really good pitches could be taken out of action because there were no preventative measures stopping these people getting on to them."
Hart Council hoped to get a court order allowing it to kick the travellers off the site by the afternoon of July 22.
But the footballer added: "What will stop them from driving back on to the site in a few days' time?
"The council really needs to take action to stop them from getting back on to the site like they did at Kennels Lane.
"This is going to be an ongoing problem because where do they go from here — the cricket green at Hartley Wintney or playing fields at Yateley? They've got to go somewhere."
Chris James, Hart Council's acting chief executive, said as soon as the travellers moved on to the site they had to assess who was there and what condition they were in under the Human Rights Act.
"We had to check if there were any young or infirm before we could move them on, which was done on Wednesday," said Mr James.
The council then applied to Aldershot Magistrates' Court for a possession order on July 17, which was heard on July 22.
"We would hope to get a bailiff and serve it straight away to move them on," said Mr James.
But one angry resident said: "It's all well and good moving them on but I really do think it is time the law is tightened up.
"How is it that these people can get away with damaging the pitches?
"If anyone else did it they would face criminal charges so why should these people be above the law?
"They are continually able to get away with it.
"It means they can turn up when they want, cause criminal damage, make the place unusable, create all sorts of havoc and then just walk away as though nothing has happened."
Our picture shows the travellers at the Peter Driver sports field in Church Crookham.