I can see in the photo that the pen arm is not exactly vertical (which is OK), but that the distal pen arm is rotated a bit from the correct position. If everything is set up perfectly, a pen that is inserted will point exactly towards the pen motor shaft.
Now, when you power on the EggBot motors, they “cog” into one of the natural positions of the motors, which are separated by about 7 degrees. These “natural” positions are not calibrated, nor necessarily the same between motors, and change as the motors turn.
If you want to center the pen precisely, then use the “walk pen motor” feature in the Manual tab of EggBot Control, to center the pen over the egg. Once you’ve walked it to vertical once, it will return to that position after each plot, until you reset the EggBot — by pressing the RST button or unplugging it from power. You may also want to figure out what that exact offset is for your machine, so that you know, for example that you should always walk the Pen Motor by (say) -30 steps when the machine is first plugged in.