Turns out it's a faulty pump. It was way louder than the old one so it sounded like it was working. I could drive the car and everything would be fine then it would hesitate and then be fine. Park up, go to start the car and it would not start.
I had the fuel pump out a few times to check everything and it looked fine. In the end I reconnected the old one and the car is back the way it was originally before I connected the new one up so has to be faulty.
The reason I was changing it was the car took a while to start but when I connected the new one it started at first turn before it started acting up.