Well... That's the main difference...
You don't want to permanently connect things that break easily, might be swapped out, have the wiring routed through other things thereby making disassembly impossible without snips.
Electrical tape is only good as a little extra insurance under shrink tube in my opinion. I try not to ever use it. Ever. Its ugly, the glue melts, it collects dirt, etc.
Also, although when PROPERLY DONE, crimping is plenty strong actually soldiering your connections is always cleaner and generally more resilient to damage or corrosion. Of course that's when done and sealed correctly, which is easy with a little practice.