NO...do not tighten it down!!! There is some thin brass in that bushing, and you can damage it! It should be adjusted so that there is only a slightly noticeable side to side "knock" in the spool when you grab it and move it from one side to the other. You can also damage the spindle for the spool. As far as it falling out...it should not do so, especially when it's tightened down. Sounds like the threads may have been stripped out...in which case, you may need a new sideplate or a new bushing (depending on which is stripped...but it's probably the sideplate).
If you have limited experience with a conventional, then, yes, you are going to get birdsnests when casting until you educate your thumb. My best advice is to load it with a heavy cheap mono (30 or 40 lb. test) and start practicing your casting with it. Once you master it, you can go to 20 or 25 lb. line and start from there. As already stated, you can also mag it, which will bring it under control. Do a search on here for "magging". I know I posted one thread on magging a Jigmaster on here, and I am sure there are others. It's pretty simple.
Also...welcome to the board!!!:beer: