When looking for good info about peppers I find call on the pros. I have a small pepper, it grows on a rather tall bush about three and a half foot high. The peppers point straight up, they start green, then turn yellow, then orange and then red. The bush looks like it is on fire. I call it my burning bush. They are only about one inch maybe one and a quarter inches. It is very hot not much flavor but very hot. The pepper is only about 3/16" round at the base and tapers to a point at the top. Please tell me what kind of a pepper I got a hold of. Thank you very much.
Without seeing a photo, I would guess that you have a Capsicum frutescens, probably a tabasco. They mature exactly in the way you describe.