If you want to grow your family quickly, have your single Nobles propose to Nobles from other families. Greet them in a Keep or Trade Post and select Propose. 

This will open a 1:1 chat with that Noble's owner and they can choose to accept or reject your proposal. If they accept, their Noble leaves their family and joins yours. The opposite happens when you're accepting a proposal instead.

Once the proposal has been accepted and you have welcomed the new Noble into your family, you can have the married couple interact with each other to make Noble babies.
This is a great way to improve your rapport with someone and in extent, make allies in the game by helping each other in boosting your families!