I found that if, you are able to change job without using all your skill points, this mess up the following job.
Because the game think you are trying to use skill point from your previous job to up your skill.
In order to prevent this i've made a modification on the job master custom NPC. He will not prevent you to change job if you have unused skill points, just asking you if you want to proceed, knowing that you will lose them forever. Then joblevel you down by your unused skill point amount right before the job change take effect.
I add my modified version here, in case someone is interested.
jobmaster.txt