Currently when your Sworn Sword levels up, you may choose to either upgrade a trait stat, or give a job specialization 1%.
Trait Stats: For example, 2 points to Battle, Trade or Intrigue. Does not scale with keep stats, will add to all three jobs separately. Example: +2 to battle will add 2 to fight, harass or aid, when those actions are taken.
Job Specialization: 1% to a specific job. Scales with keep stats, adds only to one job specialization. Example: 1% to Vanguard adds 1% of keep stats when defending or attacking with option fight. (This is great when you want to choose a Sworn Sword as defender)
Both are viable options:
Do you need immediate improvement? Choose the stat.
Do you want to plan for the long term? Specialize.
Please note: Sworn Swords do not come with you if you reincarnate unless they are permanent.