Concerns have been raised over possible changes to the use of the main sports hall at Harborough Leisure Centre.