currently in the interview process with iMPOWER consulting in a graduate role. I am wondering that since it is specifically a public sector consultancy, will this detract from my long term career potential to consult in other sectors for other firms?

I doubt it. I work for a leading UK public sector OD consultancy . iMPOWER are well respected. Theres plenty to do over the next few years re: IT / Strategy and its an area larger consultancies would like to get into. The experience of a smaller niche player will give you greater diversity and scope.