Love or not, your 9-5 is the steady paycheck, the "floor", and the starting-point... like you said, build from there, first part-time, then test out, if it fails, adapt, if it works, great, do more... take it from there and scale as you learn and grow. After this process is over, then might be a good time to think about letting go of your 9-5.