I understand your apprehension. I'm a former cancer patient, and every time I go to a doctor there is that lingering doubt in the back of your mind. Asking yourself questions like "What did they find this time?" "Are they taking longer to get back because somethings wrong with me?" that kind of thing.
However, it's important to go through with it. Fear of the unknown is common, and I sympathize with you in every way, however if you don't go and have this done, what happens if something is actually wrong with you? I don't think you want to risk not knowing. Sure a diagnosis can be scary, but once you have that, then you likely will also have a viable treatment solution and can get on with your life.
Also, on the other hand, if you don't go and it's actually nothing to worry about, you'll have in the back of your mind that nagging feeling that something is still wrong, which will likely increase your panic and anxiety.
In either case I think you should definitely go. Regardless of the outcome, it will work out better in the long run for you. Between knowing, and not knowing, I think I'd rather walk ahead in a lit hall, than fumble around in the dark.
Wishing you all the best!