Ok, I have figured it out. Normal firework stars without any dye added will not explode. The stars in the creative menues are blank and will not explode.
Yes. I used 1 paper, 1 firework star, and 1 gunpowder.
I can now confirm that it is also the same on Minecraft Bedrock Edition.
I used a firework star to make the rockets. I didn't know you needed to add dye to them to make them explode. I thought it would just explode in white or grey, It was a flawed user experience.